INFO: Downloading File to /root/PM-DARTS2/...

Succeed: Total num: 92, size: 524,678,613. OK num: 92(download 92 objects).

average speed 244036000(byte/s)

2.156520(s) elapsed
INFO: Downloading succeed.
Network is under initialization...
Network successfully initialized.
WARN: ./requirements.txt not found, skip installing requirements.
Training with a single process on 1 GPUs.
Using native Torch AMP. Training in mixed precision.
Data processing configuration for current model + dataset:
	input_size: (3, 32, 32)
	interpolation: bilinear
	mean: (0.507, 0.4865, 0.4409)
	std: (0.2673, 0.2564, 0.2761)
	crop_pct: 1.0
	crop_mode: center

-------------------------------
Learnable parameters
Student: 0.43M
Extra: 0.00M
-------------------------------
Scheduled epochs: 50
p_max: 0.4
search_space = s1
Using downloaded and verified file: /mnt/PM-DARTS2/data/cifar-100-python.tar.gz
Extracting /mnt/PM-DARTS2/data/cifar-100-python.tar.gz to /mnt/PM-DARTS2/data
Train: 0 [   0/390]  Loss: 4.770 (4.77)  Acc@1:  0.0000 ( 0.0000)  Acc@5:  4.6875 ( 4.6875)LR: 2.500e-02
Train: 0 [  50/390]  Loss: 4.160 (4.38)  Acc@1:  4.6875 ( 4.2279)  Acc@5: 23.4375 (16.8505)LR: 2.500e-02
Train: 0 [ 100/390]  Loss: 4.009 (4.22)  Acc@1:  4.6875 ( 6.1262)  Acc@5: 26.5625 (21.8131)LR: 2.500e-02
Train: 0 [ 150/390]  Loss: 4.007 (4.11)  Acc@1:  6.2500 ( 7.5021)  Acc@5: 26.5625 (24.9793)LR: 2.500e-02
Train: 0 [ 200/390]  Loss: 3.915 (4.02)  Acc@1: 10.9375 ( 8.7220)  Acc@5: 32.8125 (27.6975)LR: 2.500e-02
Train: 0 [ 250/390]  Loss: 3.693 (3.95)  Acc@1:  9.3750 ( 9.8481)  Acc@5: 35.9375 (29.9116)LR: 2.500e-02
Train: 0 [ 300/390]  Loss: 3.369 (3.88)  Acc@1: 14.0625 (10.8648)  Acc@5: 42.1875 (31.7431)LR: 2.500e-02
Train: 0 [ 350/390]  Loss: 3.670 (3.82)  Acc@1: 14.0625 (11.7610)  Acc@5: 35.9375 (33.4179)LR: 2.500e-02
Train: 0 [ 390/390]  Loss: 3.237 (3.78)  Acc@1: 22.5000 (12.3760)  Acc@5: 50.0000 (34.6800)LR: 2.500e-02
train_acc 12.376000
Valid: 0 [   0/390]  Loss: 3.432 (3.43)  Acc@1: 12.5000 (12.5000)  Acc@5: 42.1875 (42.1875)
Valid: 0 [  50/390]  Loss: 3.184 (3.28)  Acc@1: 20.3125 (19.8223)  Acc@5: 53.1250 (48.5600)
Valid: 0 [ 100/390]  Loss: 3.344 (3.30)  Acc@1: 17.1875 (19.7710)  Acc@5: 50.0000 (48.3911)
Valid: 0 [ 150/390]  Loss: 3.320 (3.31)  Acc@1: 25.0000 (19.6709)  Acc@5: 48.4375 (47.9615)
Valid: 0 [ 200/390]  Loss: 3.463 (3.31)  Acc@1: 15.6250 (19.4419)  Acc@5: 40.6250 (47.8389)
Valid: 0 [ 250/390]  Loss: 3.506 (3.31)  Acc@1: 17.1875 (19.3725)  Acc@5: 40.6250 (47.7216)
Valid: 0 [ 300/390]  Loss: 3.432 (3.32)  Acc@1: 14.0625 (19.3106)  Acc@5: 50.0000 (47.6796)
Valid: 0 [ 350/390]  Loss: 3.557 (3.32)  Acc@1: 21.8750 (19.2263)  Acc@5: 39.0625 (47.5205)
Valid: 0 [ 390/390]  Loss: 3.281 (3.32)  Acc@1: 20.0000 (19.0960)  Acc@5: 47.5000 (47.4640)
valid_acc 19.096000
epoch = 0   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('dil_conv_5x5', 0), ('dil_conv_3x3', 2), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('max_pool_3x3', 1), ('dil_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 3), ('max_pool_3x3', 1), ('max_pool_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4913, 0.5087],
        [0.4893, 0.5107],
        [0.4870, 0.5130],
        [0.4977, 0.5023],
        [0.4910, 0.5090],
        [0.5126, 0.4874],
        [0.4892, 0.5108],
        [0.4858, 0.5142],
        [0.4823, 0.5177],
        [0.4954, 0.5046],
        [0.5202, 0.4798],
        [0.4878, 0.5122],
        [0.5008, 0.4992],
        [0.5000, 0.5000]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5208, 0.4792],
        [0.5056, 0.4944],
        [0.5206, 0.4794],
        [0.5203, 0.4797],
        [0.4958, 0.5042],
        [0.5198, 0.4802],
        [0.5038, 0.4962],
        [0.5001, 0.4999],
        [0.4953, 0.5047],
        [0.5172, 0.4828],
        [0.5185, 0.4815],
        [0.5020, 0.4980],
        [0.4945, 0.5055],
        [0.4945, 0.5055]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 1 [   0/390]  Loss: 3.430 (3.43)  Acc@1: 20.3125 (20.3125)  Acc@5: 43.7500 (43.7500)LR: 2.498e-02
Train: 1 [  50/390]  Loss: 3.255 (3.24)  Acc@1: 25.0000 (20.2206)  Acc@5: 50.0000 (50.0613)LR: 2.498e-02
Train: 1 [ 100/390]  Loss: 3.174 (3.25)  Acc@1: 29.6875 (20.1578)  Acc@5: 51.5625 (49.7989)LR: 2.498e-02
Train: 1 [ 150/390]  Loss: 3.158 (3.22)  Acc@1: 18.7500 (21.0058)  Acc@5: 48.4375 (50.5795)LR: 2.498e-02
Train: 1 [ 200/390]  Loss: 2.943 (3.20)  Acc@1: 23.4375 (21.3231)  Acc@5: 59.3750 (50.9639)LR: 2.498e-02
Train: 1 [ 250/390]  Loss: 2.917 (3.16)  Acc@1: 29.6875 (21.8314)  Acc@5: 53.1250 (51.6310)LR: 2.498e-02
Train: 1 [ 300/390]  Loss: 3.135 (3.14)  Acc@1: 18.7500 (22.3474)  Acc@5: 56.2500 (52.2996)LR: 2.498e-02
Train: 1 [ 350/390]  Loss: 2.646 (3.11)  Acc@1: 32.8125 (23.0502)  Acc@5: 64.0625 (53.0404)LR: 2.498e-02
Train: 1 [ 390/390]  Loss: 2.697 (3.09)  Acc@1: 25.0000 (23.4120)  Acc@5: 55.0000 (53.6040)LR: 2.498e-02
train_acc 23.412000
Valid: 1 [   0/390]  Loss: 2.623 (2.62)  Acc@1: 39.0625 (39.0625)  Acc@5: 70.3125 (70.3125)
Valid: 1 [  50/390]  Loss: 2.916 (2.90)  Acc@1: 28.1250 (25.9191)  Acc@5: 56.2500 (58.1495)
Valid: 1 [ 100/390]  Loss: 2.979 (2.93)  Acc@1: 28.1250 (24.9226)  Acc@5: 59.3750 (57.5031)
Valid: 1 [ 150/390]  Loss: 3.211 (2.95)  Acc@1: 15.6250 (25.0621)  Acc@5: 53.1250 (57.0882)
Valid: 1 [ 200/390]  Loss: 2.949 (2.93)  Acc@1: 20.3125 (25.3887)  Acc@5: 59.3750 (57.4938)
Valid: 1 [ 250/390]  Loss: 3.191 (2.94)  Acc@1: 20.3125 (25.3548)  Acc@5: 45.3125 (57.4203)
Valid: 1 [ 300/390]  Loss: 2.502 (2.93)  Acc@1: 32.8125 (25.5970)  Acc@5: 73.4375 (57.5997)
Valid: 1 [ 350/390]  Loss: 3.051 (2.93)  Acc@1: 20.3125 (25.7746)  Acc@5: 56.2500 (57.8036)
Valid: 1 [ 390/390]  Loss: 2.898 (2.93)  Acc@1: 22.5000 (25.6120)  Acc@5: 65.0000 (57.6560)
valid_acc 25.612000
epoch = 1   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('dil_conv_5x5', 0), ('dil_conv_3x3', 2), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('max_pool_3x3', 1), ('dil_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('max_pool_3x3', 0), ('dil_conv_3x3', 2), ('max_pool_3x3', 1), ('max_pool_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4836, 0.5164],
        [0.4702, 0.5298],
        [0.4749, 0.5251],
        [0.4888, 0.5112],
        [0.4768, 0.5232],
        [0.5265, 0.4735],
        [0.4754, 0.5246],
        [0.4689, 0.5311],
        [0.4617, 0.5383],
        [0.4944, 0.5056],
        [0.5363, 0.4637],
        [0.4725, 0.5275],
        [0.4897, 0.5103],
        [0.4931, 0.5069]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5370, 0.4630],
        [0.5153, 0.4847],
        [0.5351, 0.4649],
        [0.5368, 0.4632],
        [0.4774, 0.5226],
        [0.5353, 0.4647],
        [0.5075, 0.4925],
        [0.4900, 0.5100],
        [0.4916, 0.5084],
        [0.5309, 0.4691],
        [0.5339, 0.4661],
        [0.4871, 0.5129],
        [0.4836, 0.5164],
        [0.4886, 0.5114]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 2 [   0/390]  Loss: 3.055 (3.06)  Acc@1: 23.4375 (23.4375)  Acc@5: 51.5625 (51.5625)LR: 2.491e-02
Train: 2 [  50/390]  Loss: 2.760 (2.85)  Acc@1: 28.1250 (27.9105)  Acc@5: 59.3750 (59.9877)LR: 2.491e-02
Train: 2 [ 100/390]  Loss: 3.103 (2.84)  Acc@1: 23.4375 (28.0786)  Acc@5: 51.5625 (60.1176)LR: 2.491e-02
Train: 2 [ 150/390]  Loss: 2.705 (2.83)  Acc@1: 35.9375 (28.5286)  Acc@5: 60.9375 (60.2546)LR: 2.491e-02
Train: 2 [ 200/390]  Loss: 2.074 (2.80)  Acc@1: 45.3125 (29.1900)  Acc@5: 81.2500 (61.0152)LR: 2.491e-02
Train: 2 [ 250/390]  Loss: 2.633 (2.77)  Acc@1: 26.5625 (29.5070)  Acc@5: 68.7500 (61.6160)LR: 2.491e-02
Train: 2 [ 300/390]  Loss: 2.472 (2.76)  Acc@1: 26.5625 (29.5370)  Acc@5: 68.7500 (61.8667)LR: 2.491e-02
Train: 2 [ 350/390]  Loss: 2.326 (2.75)  Acc@1: 29.6875 (29.6474)  Acc@5: 71.8750 (62.1661)LR: 2.491e-02
Train: 2 [ 390/390]  Loss: 2.035 (2.74)  Acc@1: 50.0000 (29.8800)  Acc@5: 70.0000 (62.3840)LR: 2.491e-02
train_acc 29.880000
Valid: 2 [   0/390]  Loss: 2.666 (2.67)  Acc@1: 34.3750 (34.3750)  Acc@5: 57.8125 (57.8125)
Valid: 2 [  50/390]  Loss: 2.857 (2.69)  Acc@1: 20.3125 (31.3419)  Acc@5: 54.6875 (62.9596)
Valid: 2 [ 100/390]  Loss: 2.826 (2.72)  Acc@1: 28.1250 (30.9715)  Acc@5: 59.3750 (62.4691)
Valid: 2 [ 150/390]  Loss: 2.906 (2.74)  Acc@1: 32.8125 (30.4739)  Acc@5: 64.0625 (62.2620)
Valid: 2 [ 200/390]  Loss: 2.820 (2.74)  Acc@1: 34.3750 (30.4571)  Acc@5: 60.9375 (62.3756)
Valid: 2 [ 250/390]  Loss: 3.059 (2.73)  Acc@1: 26.5625 (30.5652)  Acc@5: 59.3750 (62.5934)
Valid: 2 [ 300/390]  Loss: 2.795 (2.73)  Acc@1: 25.0000 (30.5077)  Acc@5: 62.5000 (62.5779)
Valid: 2 [ 350/390]  Loss: 2.857 (2.73)  Acc@1: 26.5625 (30.7959)  Acc@5: 59.3750 (62.6335)
Valid: 2 [ 390/390]  Loss: 2.623 (2.73)  Acc@1: 30.0000 (30.9240)  Acc@5: 62.5000 (62.6640)
valid_acc 30.924000
epoch = 2   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('dil_conv_5x5', 0), ('dil_conv_3x3', 2), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('max_pool_3x3', 1), ('dil_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('max_pool_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 0), ('dil_conv_5x5', 3), ('max_pool_3x3', 1), ('max_pool_3x3', 0)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4642, 0.5358],
        [0.4446, 0.5554],
        [0.4512, 0.5488],
        [0.4628, 0.5372],
        [0.4516, 0.5484],
        [0.5394, 0.4606],
        [0.4445, 0.5555],
        [0.4480, 0.5520],
        [0.4371, 0.5629],
        [0.4871, 0.5129],
        [0.5532, 0.4468],
        [0.4548, 0.5452],
        [0.4824, 0.5176],
        [0.4816, 0.5184]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5459, 0.4541],
        [0.5198, 0.4802],
        [0.5434, 0.4566],
        [0.5464, 0.4536],
        [0.4565, 0.5435],
        [0.5455, 0.4545],
        [0.5089, 0.4911],
        [0.4763, 0.5237],
        [0.4723, 0.5277],
        [0.5391, 0.4609],
        [0.5445, 0.4555],
        [0.4667, 0.5333],
        [0.4698, 0.5302],
        [0.4765, 0.5235]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 3 [   0/390]  Loss: 2.550 (2.55)  Acc@1: 35.9375 (35.9375)  Acc@5: 68.7500 (68.7500)LR: 2.479e-02
Train: 3 [  50/390]  Loss: 2.556 (2.50)  Acc@1: 28.1250 (34.6201)  Acc@5: 68.7500 (68.2904)LR: 2.479e-02
Train: 3 [ 100/390]  Loss: 2.802 (2.53)  Acc@1: 23.4375 (34.1894)  Acc@5: 53.1250 (67.0947)LR: 2.479e-02
Train: 3 [ 150/390]  Loss: 2.343 (2.51)  Acc@1: 39.0625 (34.7475)  Acc@5: 75.0000 (67.6325)LR: 2.479e-02
Train: 3 [ 200/390]  Loss: 2.572 (2.51)  Acc@1: 37.5000 (34.6859)  Acc@5: 67.1875 (67.5995)LR: 2.479e-02
Train: 3 [ 250/390]  Loss: 2.761 (2.51)  Acc@1: 32.8125 (34.6365)  Acc@5: 60.9375 (67.4801)LR: 2.479e-02
Train: 3 [ 300/390]  Loss: 2.438 (2.50)  Acc@1: 39.0625 (34.8889)  Acc@5: 68.7500 (67.7949)LR: 2.479e-02
Train: 3 [ 350/390]  Loss: 2.377 (2.49)  Acc@1: 37.5000 (35.1807)  Acc@5: 68.7500 (67.9398)LR: 2.479e-02
Train: 3 [ 390/390]  Loss: 2.990 (2.48)  Acc@1: 32.5000 (35.2200)  Acc@5: 50.0000 (68.0000)LR: 2.479e-02
train_acc 35.220000
Valid: 3 [   0/390]  Loss: 2.436 (2.44)  Acc@1: 37.5000 (37.5000)  Acc@5: 70.3125 (70.3125)
Valid: 3 [  50/390]  Loss: 2.686 (2.57)  Acc@1: 29.6875 (35.1103)  Acc@5: 65.6250 (66.4522)
Valid: 3 [ 100/390]  Loss: 2.754 (2.59)  Acc@1: 35.9375 (34.3131)  Acc@5: 64.0625 (65.9344)
Valid: 3 [ 150/390]  Loss: 2.783 (2.59)  Acc@1: 31.2500 (34.1680)  Acc@5: 56.2500 (65.8009)
Valid: 3 [ 200/390]  Loss: 2.910 (2.59)  Acc@1: 31.2500 (34.4994)  Acc@5: 59.3750 (65.9904)
Valid: 3 [ 250/390]  Loss: 2.834 (2.61)  Acc@1: 25.0000 (34.1384)  Acc@5: 57.8125 (65.8242)
Valid: 3 [ 300/390]  Loss: 2.375 (2.61)  Acc@1: 37.5000 (34.1674)  Acc@5: 70.3125 (65.8949)
Valid: 3 [ 350/390]  Loss: 2.316 (2.61)  Acc@1: 37.5000 (34.1524)  Acc@5: 70.3125 (65.9411)
Valid: 3 [ 390/390]  Loss: 2.674 (2.61)  Acc@1: 22.5000 (34.1720)  Acc@5: 70.0000 (65.9960)
valid_acc 34.172000
epoch = 3   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('dil_conv_5x5', 0), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2), ('max_pool_3x3', 1)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4427, 0.5573],
        [0.4201, 0.5799],
        [0.4296, 0.5704],
        [0.4371, 0.5629],
        [0.4316, 0.5684],
        [0.5462, 0.4538],
        [0.4135, 0.5865],
        [0.4269, 0.5731],
        [0.4160, 0.5840],
        [0.4752, 0.5248],
        [0.5628, 0.4372],
        [0.4370, 0.5630],
        [0.4817, 0.5183],
        [0.4712, 0.5288]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5569, 0.4431],
        [0.5153, 0.4847],
        [0.5502, 0.4498],
        [0.5516, 0.4484],
        [0.4299, 0.5701],
        [0.5537, 0.4463],
        [0.5022, 0.4978],
        [0.4579, 0.5421],
        [0.4522, 0.5478],
        [0.5449, 0.4551],
        [0.5500, 0.4500],
        [0.4397, 0.5603],
        [0.4488, 0.5512],
        [0.4549, 0.5451]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 4 [   0/390]  Loss: 2.149 (2.15)  Acc@1: 34.3750 (34.3750)  Acc@5: 79.6875 (79.6875)LR: 2.462e-02
Train: 4 [  50/390]  Loss: 2.557 (2.38)  Acc@1: 31.2500 (37.4694)  Acc@5: 64.0625 (70.5882)LR: 2.462e-02
Train: 4 [ 100/390]  Loss: 2.094 (2.34)  Acc@1: 51.5625 (38.2271)  Acc@5: 79.6875 (71.3335)LR: 2.462e-02
Train: 4 [ 150/390]  Loss: 2.355 (2.34)  Acc@1: 37.5000 (38.4002)  Acc@5: 67.1875 (71.2334)LR: 2.462e-02
Train: 4 [ 200/390]  Loss: 2.276 (2.32)  Acc@1: 42.1875 (38.7516)  Acc@5: 73.4375 (71.4941)LR: 2.462e-02
Train: 4 [ 250/390]  Loss: 2.160 (2.32)  Acc@1: 35.9375 (39.0002)  Acc@5: 79.6875 (71.4890)LR: 2.462e-02
Train: 4 [ 300/390]  Loss: 2.314 (2.32)  Acc@1: 35.9375 (39.0054)  Acc@5: 71.8750 (71.5376)LR: 2.462e-02
Train: 4 [ 350/390]  Loss: 2.226 (2.31)  Acc@1: 37.5000 (39.1293)  Acc@5: 76.5625 (71.5812)LR: 2.462e-02
Train: 4 [ 390/390]  Loss: 2.075 (2.30)  Acc@1: 45.0000 (39.3000)  Acc@5: 77.5000 (71.9120)LR: 2.462e-02
train_acc 39.300000
Valid: 4 [   0/390]  Loss: 2.160 (2.16)  Acc@1: 39.0625 (39.0625)  Acc@5: 67.1875 (67.1875)
Valid: 4 [  50/390]  Loss: 2.309 (2.37)  Acc@1: 37.5000 (39.1544)  Acc@5: 78.1250 (70.8333)
Valid: 4 [ 100/390]  Loss: 2.072 (2.34)  Acc@1: 43.7500 (39.4647)  Acc@5: 79.6875 (71.8286)
Valid: 4 [ 150/390]  Loss: 2.350 (2.33)  Acc@1: 32.8125 (39.6213)  Acc@5: 65.6250 (72.0406)
Valid: 4 [ 200/390]  Loss: 2.242 (2.33)  Acc@1: 39.0625 (39.7466)  Acc@5: 79.6875 (72.1160)
Valid: 4 [ 250/390]  Loss: 2.336 (2.33)  Acc@1: 35.9375 (39.8593)  Acc@5: 68.7500 (72.0244)
Valid: 4 [ 300/390]  Loss: 2.988 (2.34)  Acc@1: 32.8125 (39.4985)  Acc@5: 53.1250 (71.5895)
Valid: 4 [ 350/390]  Loss: 2.979 (2.35)  Acc@1: 37.5000 (39.6679)  Acc@5: 65.6250 (71.6702)
Valid: 4 [ 390/390]  Loss: 2.031 (2.35)  Acc@1: 37.5000 (39.6160)  Acc@5: 82.5000 (71.5480)
valid_acc 39.616000
epoch = 4   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('dil_conv_5x5', 0), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2), ('max_pool_3x3', 1)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.4177, 0.5823],
        [0.3978, 0.6022],
        [0.4079, 0.5921],
        [0.4151, 0.5849],
        [0.4141, 0.5859],
        [0.5403, 0.4597],
        [0.3822, 0.6178],
        [0.4022, 0.5978],
        [0.3919, 0.6081],
        [0.4628, 0.5372],
        [0.5675, 0.4325],
        [0.4126, 0.5874],
        [0.4787, 0.5213],
        [0.4698, 0.5302]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5591, 0.4409],
        [0.5135, 0.4865],
        [0.5496, 0.4504],
        [0.5535, 0.4465],
        [0.4022, 0.5978],
        [0.5536, 0.4464],
        [0.4926, 0.5074],
        [0.4361, 0.5639],
        [0.4272, 0.5728],
        [0.5431, 0.4569],
        [0.5525, 0.4475],
        [0.4062, 0.5938],
        [0.4234, 0.5766],
        [0.4341, 0.5659]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 5 [   0/390]  Loss: 1.927 (1.93)  Acc@1: 43.7500 (43.7500)  Acc@5: 82.8125 (82.8125)LR: 2.441e-02
Train: 5 [  50/390]  Loss: 1.906 (2.16)  Acc@1: 42.1875 (41.6973)  Acc@5: 79.6875 (75.5515)LR: 2.441e-02
Train: 5 [ 100/390]  Loss: 1.656 (2.18)  Acc@1: 54.6875 (41.5842)  Acc@5: 85.9375 (74.3193)LR: 2.441e-02
Train: 5 [ 150/390]  Loss: 2.351 (2.18)  Acc@1: 31.2500 (41.6391)  Acc@5: 76.5625 (74.4309)LR: 2.441e-02
Train: 5 [ 200/390]  Loss: 1.872 (2.17)  Acc@1: 48.4375 (41.9387)  Acc@5: 78.1250 (74.5569)LR: 2.441e-02
Train: 5 [ 250/390]  Loss: 2.456 (2.16)  Acc@1: 31.2500 (42.2747)  Acc@5: 70.3125 (74.6016)LR: 2.441e-02
Train: 5 [ 300/390]  Loss: 2.170 (2.16)  Acc@1: 45.3125 (42.1564)  Acc@5: 70.3125 (74.6522)LR: 2.441e-02
Train: 5 [ 350/390]  Loss: 2.385 (2.16)  Acc@1: 34.3750 (42.1697)  Acc@5: 68.7500 (74.7908)LR: 2.441e-02
Train: 5 [ 390/390]  Loss: 2.260 (2.16)  Acc@1: 35.0000 (42.3040)  Acc@5: 80.0000 (74.8440)LR: 2.441e-02
train_acc 42.304000
Valid: 5 [   0/390]  Loss: 2.250 (2.25)  Acc@1: 34.3750 (34.3750)  Acc@5: 75.0000 (75.0000)
Valid: 5 [  50/390]  Loss: 2.264 (2.21)  Acc@1: 43.7500 (40.5637)  Acc@5: 71.8750 (74.4485)
Valid: 5 [ 100/390]  Loss: 1.949 (2.21)  Acc@1: 48.4375 (40.9653)  Acc@5: 75.0000 (74.3348)
Valid: 5 [ 150/390]  Loss: 2.074 (2.23)  Acc@1: 46.8750 (41.2459)  Acc@5: 75.0000 (74.2343)
Valid: 5 [ 200/390]  Loss: 2.238 (2.21)  Acc@1: 35.9375 (41.7055)  Acc@5: 71.8750 (74.2304)
Valid: 5 [ 250/390]  Loss: 2.111 (2.22)  Acc@1: 48.4375 (41.8140)  Acc@5: 76.5625 (73.9915)
Valid: 5 [ 300/390]  Loss: 1.660 (2.21)  Acc@1: 54.6875 (42.0577)  Acc@5: 79.6875 (74.1331)
Valid: 5 [ 350/390]  Loss: 2.367 (2.21)  Acc@1: 37.5000 (42.1118)  Acc@5: 73.4375 (74.1631)
Valid: 5 [ 390/390]  Loss: 2.525 (2.21)  Acc@1: 32.5000 (42.2040)  Acc@5: 67.5000 (74.2360)
valid_acc 42.204000
epoch = 5   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('dil_conv_5x5', 0), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2), ('max_pool_3x3', 1)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3902, 0.6098],
        [0.3770, 0.6230],
        [0.3879, 0.6121],
        [0.3954, 0.6046],
        [0.3952, 0.6048],
        [0.5378, 0.4622],
        [0.3534, 0.6466],
        [0.3819, 0.6181],
        [0.3662, 0.6338],
        [0.4413, 0.5587],
        [0.5738, 0.4262],
        [0.3922, 0.6078],
        [0.4782, 0.5218],
        [0.4706, 0.5294]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5602, 0.4398],
        [0.5106, 0.4894],
        [0.5480, 0.4520],
        [0.5529, 0.4471],
        [0.3769, 0.6231],
        [0.5525, 0.4475],
        [0.4823, 0.5177],
        [0.4138, 0.5862],
        [0.4049, 0.5951],
        [0.5387, 0.4613],
        [0.5512, 0.4488],
        [0.3778, 0.6222],
        [0.4027, 0.5973],
        [0.4139, 0.5861]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 6 [   0/390]  Loss: 1.817 (1.82)  Acc@1: 48.4375 (48.4375)  Acc@5: 81.2500 (81.2500)LR: 2.416e-02
Train: 6 [  50/390]  Loss: 1.764 (1.97)  Acc@1: 48.4375 (45.8027)  Acc@5: 82.8125 (78.8297)LR: 2.416e-02
Train: 6 [ 100/390]  Loss: 2.280 (2.00)  Acc@1: 43.7500 (45.5446)  Acc@5: 73.4375 (77.8775)LR: 2.416e-02
Train: 6 [ 150/390]  Loss: 1.947 (2.02)  Acc@1: 37.5000 (45.1159)  Acc@5: 82.8125 (77.6076)LR: 2.416e-02
Train: 6 [ 200/390]  Loss: 2.032 (2.03)  Acc@1: 45.3125 (44.9160)  Acc@5: 81.2500 (77.2233)LR: 2.416e-02
Train: 6 [ 250/390]  Loss: 2.243 (2.04)  Acc@1: 45.3125 (44.9577)  Acc@5: 71.8750 (76.9983)LR: 2.416e-02
Train: 6 [ 300/390]  Loss: 1.945 (2.04)  Acc@1: 45.3125 (45.0633)  Acc@5: 79.6875 (77.0608)LR: 2.416e-02
Train: 6 [ 350/390]  Loss: 2.061 (2.03)  Acc@1: 42.1875 (45.0499)  Acc@5: 78.1250 (77.1768)LR: 2.416e-02
Train: 6 [ 390/390]  Loss: 1.695 (2.03)  Acc@1: 50.0000 (45.2080)  Acc@5: 80.0000 (77.3480)LR: 2.416e-02
train_acc 45.208000
Valid: 6 [   0/390]  Loss: 1.983 (1.98)  Acc@1: 51.5625 (51.5625)  Acc@5: 75.0000 (75.0000)
Valid: 6 [  50/390]  Loss: 2.023 (2.17)  Acc@1: 46.8750 (43.1373)  Acc@5: 75.0000 (74.9694)
Valid: 6 [ 100/390]  Loss: 2.365 (2.18)  Acc@1: 35.9375 (43.2085)  Acc@5: 73.4375 (75.4332)
Valid: 6 [ 150/390]  Loss: 2.137 (2.20)  Acc@1: 45.3125 (42.8291)  Acc@5: 76.5625 (75.1138)
Valid: 6 [ 200/390]  Loss: 2.057 (2.22)  Acc@1: 48.4375 (42.5917)  Acc@5: 76.5625 (74.7201)
Valid: 6 [ 250/390]  Loss: 1.844 (2.21)  Acc@1: 53.1250 (42.6482)  Acc@5: 84.3750 (74.6016)
Valid: 6 [ 300/390]  Loss: 2.053 (2.20)  Acc@1: 51.5625 (42.5768)  Acc@5: 79.6875 (74.7197)
Valid: 6 [ 350/390]  Loss: 2.262 (2.20)  Acc@1: 46.8750 (42.5080)  Acc@5: 78.1250 (74.6528)
Valid: 6 [ 390/390]  Loss: 1.884 (2.20)  Acc@1: 47.5000 (42.6240)  Acc@5: 75.0000 (74.7400)
valid_acc 42.624000
epoch = 6   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('dil_conv_5x5', 0), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('max_pool_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3707, 0.6293],
        [0.3589, 0.6411],
        [0.3706, 0.6294],
        [0.3804, 0.6196],
        [0.3773, 0.6227],
        [0.5276, 0.4724],
        [0.3342, 0.6658],
        [0.3642, 0.6358],
        [0.3465, 0.6535],
        [0.4207, 0.5793],
        [0.5704, 0.4296],
        [0.3733, 0.6267],
        [0.4786, 0.5214],
        [0.4706, 0.5294]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5587, 0.4413],
        [0.5050, 0.4950],
        [0.5425, 0.4575],
        [0.5551, 0.4449],
        [0.3532, 0.6468],
        [0.5512, 0.4488],
        [0.4729, 0.5271],
        [0.3924, 0.6076],
        [0.3775, 0.6225],
        [0.5328, 0.4672],
        [0.5511, 0.4489],
        [0.3535, 0.6465],
        [0.3828, 0.6172],
        [0.3985, 0.6015]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 7 [   0/390]  Loss: 1.962 (1.96)  Acc@1: 43.7500 (43.7500)  Acc@5: 76.5625 (76.5625)LR: 2.386e-02
Train: 7 [  50/390]  Loss: 1.770 (1.88)  Acc@1: 48.4375 (48.0086)  Acc@5: 82.8125 (80.4228)LR: 2.386e-02
Train: 7 [ 100/390]  Loss: 1.811 (1.89)  Acc@1: 48.4375 (47.6330)  Acc@5: 85.9375 (80.0433)LR: 2.386e-02
Train: 7 [ 150/390]  Loss: 2.016 (1.91)  Acc@1: 32.8125 (47.2889)  Acc@5: 82.8125 (79.8634)LR: 2.386e-02
Train: 7 [ 200/390]  Loss: 2.315 (1.92)  Acc@1: 28.1250 (47.0149)  Acc@5: 76.5625 (79.3221)LR: 2.386e-02
Train: 7 [ 250/390]  Loss: 1.729 (1.92)  Acc@1: 50.0000 (47.0057)  Acc@5: 81.2500 (79.0650)LR: 2.386e-02
Train: 7 [ 300/390]  Loss: 1.947 (1.93)  Acc@1: 43.7500 (46.9684)  Acc@5: 79.6875 (78.7843)LR: 2.386e-02
Train: 7 [ 350/390]  Loss: 1.974 (1.92)  Acc@1: 40.6250 (47.3291)  Acc@5: 79.6875 (78.9975)LR: 2.386e-02
Train: 7 [ 390/390]  Loss: 2.211 (1.92)  Acc@1: 47.5000 (47.3720)  Acc@5: 70.0000 (79.0200)LR: 2.386e-02
train_acc 47.372000
Valid: 7 [   0/390]  Loss: 2.578 (2.58)  Acc@1: 40.6250 (40.6250)  Acc@5: 68.7500 (68.7500)
Valid: 7 [  50/390]  Loss: 2.053 (2.06)  Acc@1: 46.8750 (45.3431)  Acc@5: 75.0000 (76.9301)
Valid: 7 [ 100/390]  Loss: 1.864 (2.05)  Acc@1: 46.8750 (46.2098)  Acc@5: 76.5625 (77.1040)
Valid: 7 [ 150/390]  Loss: 2.270 (2.06)  Acc@1: 42.1875 (45.9851)  Acc@5: 70.3125 (76.9661)
Valid: 7 [ 200/390]  Loss: 1.709 (2.06)  Acc@1: 46.8750 (45.9033)  Acc@5: 87.5000 (77.2699)
Valid: 7 [ 250/390]  Loss: 2.293 (2.07)  Acc@1: 43.7500 (45.9412)  Acc@5: 75.0000 (77.1352)
Valid: 7 [ 300/390]  Loss: 2.174 (2.07)  Acc@1: 42.1875 (45.8783)  Acc@5: 73.4375 (76.8688)
Valid: 7 [ 350/390]  Loss: 2.229 (2.07)  Acc@1: 42.1875 (45.7354)  Acc@5: 73.4375 (76.9453)
Valid: 7 [ 390/390]  Loss: 2.092 (2.08)  Acc@1: 45.0000 (45.6320)  Acc@5: 82.5000 (76.9000)
valid_acc 45.632000
epoch = 7   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('dil_conv_5x5', 0), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3478, 0.6522],
        [0.3413, 0.6587],
        [0.3535, 0.6465],
        [0.3607, 0.6393],
        [0.3606, 0.6394],
        [0.5166, 0.4834],
        [0.3092, 0.6908],
        [0.3406, 0.6594],
        [0.3223, 0.6777],
        [0.4044, 0.5956],
        [0.5679, 0.4321],
        [0.3576, 0.6424],
        [0.4806, 0.5194],
        [0.4735, 0.5265]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5533, 0.4467],
        [0.4853, 0.5147],
        [0.5367, 0.4633],
        [0.5524, 0.4476],
        [0.3326, 0.6674],
        [0.5466, 0.4534],
        [0.4563, 0.5437],
        [0.3736, 0.6264],
        [0.3516, 0.6484],
        [0.5270, 0.4730],
        [0.5484, 0.4516],
        [0.3294, 0.6706],
        [0.3563, 0.6437],
        [0.3822, 0.6178]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 8 [   0/390]  Loss: 1.814 (1.81)  Acc@1: 48.4375 (48.4375)  Acc@5: 81.2500 (81.2500)LR: 2.352e-02
Train: 8 [  50/390]  Loss: 1.943 (1.76)  Acc@1: 53.1250 (51.9608)  Acc@5: 73.4375 (82.1385)LR: 2.352e-02
Train: 8 [ 100/390]  Loss: 1.738 (1.77)  Acc@1: 57.8125 (51.6708)  Acc@5: 78.1250 (81.7605)LR: 2.352e-02
Train: 8 [ 150/390]  Loss: 1.822 (1.80)  Acc@1: 60.9375 (50.9934)  Acc@5: 79.6875 (81.2914)LR: 2.352e-02
Train: 8 [ 200/390]  Loss: 2.086 (1.81)  Acc@1: 48.4375 (50.4198)  Acc@5: 75.0000 (81.1334)LR: 2.352e-02
Train: 8 [ 250/390]  Loss: 1.773 (1.82)  Acc@1: 51.5625 (50.0685)  Acc@5: 87.5000 (80.9450)LR: 2.352e-02
Train: 8 [ 300/390]  Loss: 2.029 (1.83)  Acc@1: 42.1875 (49.7456)  Acc@5: 71.8750 (80.8814)LR: 2.352e-02
Train: 8 [ 350/390]  Loss: 1.907 (1.83)  Acc@1: 43.7500 (49.6795)  Acc@5: 79.6875 (80.8983)LR: 2.352e-02
Train: 8 [ 390/390]  Loss: 2.625 (1.83)  Acc@1: 35.0000 (49.6760)  Acc@5: 65.0000 (80.8840)LR: 2.352e-02
train_acc 49.676000
Valid: 8 [   0/390]  Loss: 1.605 (1.61)  Acc@1: 50.0000 (50.0000)  Acc@5: 85.9375 (85.9375)
Valid: 8 [  50/390]  Loss: 1.998 (2.10)  Acc@1: 42.1875 (46.0784)  Acc@5: 81.2500 (77.0527)
Valid: 8 [ 100/390]  Loss: 2.395 (2.06)  Acc@1: 42.1875 (46.5965)  Acc@5: 65.6250 (78.2642)
Valid: 8 [ 150/390]  Loss: 2.154 (2.06)  Acc@1: 48.4375 (46.6163)  Acc@5: 78.1250 (78.2078)
Valid: 8 [ 200/390]  Loss: 2.258 (2.06)  Acc@1: 43.7500 (46.6262)  Acc@5: 79.6875 (78.0006)
Valid: 8 [ 250/390]  Loss: 2.174 (2.07)  Acc@1: 40.6250 (46.3521)  Acc@5: 73.4375 (77.7951)
Valid: 8 [ 300/390]  Loss: 2.262 (2.07)  Acc@1: 43.7500 (46.5635)  Acc@5: 75.0000 (77.7149)
Valid: 8 [ 350/390]  Loss: 1.812 (2.07)  Acc@1: 54.6875 (46.5589)  Acc@5: 79.6875 (77.5908)
Valid: 8 [ 390/390]  Loss: 1.918 (2.06)  Acc@1: 45.0000 (46.5720)  Acc@5: 77.5000 (77.7080)
valid_acc 46.572000
epoch = 8   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3269, 0.6731],
        [0.3205, 0.6795],
        [0.3337, 0.6663],
        [0.3427, 0.6573],
        [0.3451, 0.6549],
        [0.5041, 0.4959],
        [0.2881, 0.7119],
        [0.3226, 0.6774],
        [0.3029, 0.6971],
        [0.3918, 0.6082],
        [0.5623, 0.4377],
        [0.3440, 0.6560],
        [0.4832, 0.5168],
        [0.4741, 0.5259]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5516, 0.4484],
        [0.4717, 0.5283],
        [0.5346, 0.4654],
        [0.5513, 0.4487],
        [0.3088, 0.6912],
        [0.5475, 0.4525],
        [0.4403, 0.5597],
        [0.3551, 0.6449],
        [0.3289, 0.6711],
        [0.5238, 0.4762],
        [0.5458, 0.4542],
        [0.3067, 0.6933],
        [0.3369, 0.6631],
        [0.3708, 0.6292]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 9 [   0/390]  Loss: 1.736 (1.74)  Acc@1: 56.2500 (56.2500)  Acc@5: 84.3750 (84.3750)LR: 2.313e-02
Train: 9 [  50/390]  Loss: 1.819 (1.73)  Acc@1: 54.6875 (51.5319)  Acc@5: 81.2500 (83.2108)LR: 2.313e-02
Train: 9 [ 100/390]  Loss: 1.904 (1.74)  Acc@1: 43.7500 (51.3614)  Acc@5: 79.6875 (82.9981)LR: 2.313e-02
Train: 9 [ 150/390]  Loss: 2.164 (1.76)  Acc@1: 42.1875 (51.0141)  Acc@5: 76.5625 (82.3055)LR: 2.313e-02
Train: 9 [ 200/390]  Loss: 2.048 (1.76)  Acc@1: 42.1875 (51.0883)  Acc@5: 79.6875 (82.5016)LR: 2.313e-02
Train: 9 [ 250/390]  Loss: 1.841 (1.75)  Acc@1: 50.0000 (51.0894)  Acc@5: 71.8750 (82.5137)LR: 2.313e-02
Train: 9 [ 300/390]  Loss: 1.626 (1.75)  Acc@1: 48.4375 (51.0226)  Acc@5: 82.8125 (82.4699)LR: 2.313e-02
Train: 9 [ 350/390]  Loss: 1.685 (1.75)  Acc@1: 50.0000 (50.9838)  Acc@5: 78.1250 (82.3629)LR: 2.313e-02
Train: 9 [ 390/390]  Loss: 1.586 (1.76)  Acc@1: 57.5000 (50.9240)  Acc@5: 85.0000 (82.3240)LR: 2.313e-02
train_acc 50.924000
Valid: 9 [   0/390]  Loss: 2.096 (2.10)  Acc@1: 50.0000 (50.0000)  Acc@5: 79.6875 (79.6875)
Valid: 9 [  50/390]  Loss: 1.861 (2.14)  Acc@1: 51.5625 (45.2819)  Acc@5: 76.5625 (76.8995)
Valid: 9 [ 100/390]  Loss: 2.582 (2.16)  Acc@1: 31.2500 (45.4208)  Acc@5: 73.4375 (76.3304)
Valid: 9 [ 150/390]  Loss: 2.111 (2.12)  Acc@1: 50.0000 (46.3266)  Acc@5: 81.2500 (76.8005)
Valid: 9 [ 200/390]  Loss: 2.535 (2.11)  Acc@1: 39.0625 (46.2609)  Acc@5: 68.7500 (76.9356)
Valid: 9 [ 250/390]  Loss: 2.594 (2.11)  Acc@1: 37.5000 (46.2712)  Acc@5: 65.6250 (76.9173)
Valid: 9 [ 300/390]  Loss: 2.318 (2.10)  Acc@1: 45.3125 (46.2728)  Acc@5: 70.3125 (77.0868)
Valid: 9 [ 350/390]  Loss: 1.854 (2.11)  Acc@1: 46.8750 (46.1984)  Acc@5: 81.2500 (77.0611)
Valid: 9 [ 390/390]  Loss: 2.029 (2.11)  Acc@1: 50.0000 (46.1280)  Acc@5: 77.5000 (76.9200)
valid_acc 46.128000
epoch = 9   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('max_pool_3x3', 0), ('dil_conv_3x3', 1), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.3104, 0.6896],
        [0.3039, 0.6961],
        [0.3184, 0.6816],
        [0.3267, 0.6733],
        [0.3285, 0.6715],
        [0.4862, 0.5138],
        [0.2689, 0.7311],
        [0.3024, 0.6976],
        [0.2820, 0.7180],
        [0.3771, 0.6229],
        [0.5488, 0.4512],
        [0.3329, 0.6671],
        [0.4862, 0.5138],
        [0.4789, 0.5211]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5499, 0.4501],
        [0.4583, 0.5417],
        [0.5310, 0.4690],
        [0.5530, 0.4470],
        [0.2886, 0.7114],
        [0.5442, 0.4558],
        [0.4271, 0.5729],
        [0.3381, 0.6619],
        [0.3092, 0.6908],
        [0.5191, 0.4809],
        [0.5473, 0.4527],
        [0.2863, 0.7137],
        [0.3154, 0.6846],
        [0.3542, 0.6458]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 10 [   0/390]  Loss: 1.630 (1.63)  Acc@1: 51.5625 (51.5625)  Acc@5: 92.1875 (92.1875)LR: 2.271e-02
Train: 10 [  50/390]  Loss: 1.897 (1.63)  Acc@1: 51.5625 (54.6875)  Acc@5: 82.8125 (84.3137)LR: 2.271e-02
Train: 10 [ 100/390]  Loss: 1.774 (1.63)  Acc@1: 57.8125 (55.0743)  Acc@5: 78.1250 (84.1275)LR: 2.271e-02
Train: 10 [ 150/390]  Loss: 1.743 (1.66)  Acc@1: 45.3125 (54.0149)  Acc@5: 73.4375 (83.7127)LR: 2.271e-02
Train: 10 [ 200/390]  Loss: 1.488 (1.66)  Acc@1: 65.6250 (53.7313)  Acc@5: 82.8125 (83.6443)LR: 2.271e-02
Train: 10 [ 250/390]  Loss: 1.802 (1.68)  Acc@1: 45.3125 (53.3242)  Acc@5: 85.9375 (83.3416)LR: 2.271e-02
Train: 10 [ 300/390]  Loss: 1.247 (1.70)  Acc@1: 62.5000 (52.9745)  Acc@5: 90.6250 (83.1551)LR: 2.271e-02
Train: 10 [ 350/390]  Loss: 2.265 (1.69)  Acc@1: 39.0625 (52.9959)  Acc@5: 76.5625 (83.3066)LR: 2.271e-02
Train: 10 [ 390/390]  Loss: 1.623 (1.69)  Acc@1: 65.0000 (52.9640)  Acc@5: 82.5000 (83.2600)LR: 2.271e-02
train_acc 52.964000
Valid: 10 [   0/390]  Loss: 1.764 (1.76)  Acc@1: 54.6875 (54.6875)  Acc@5: 81.2500 (81.2500)
Valid: 10 [  50/390]  Loss: 1.997 (2.06)  Acc@1: 50.0000 (48.6213)  Acc@5: 79.6875 (78.6152)
Valid: 10 [ 100/390]  Loss: 2.627 (2.06)  Acc@1: 39.0625 (47.7723)  Acc@5: 71.8750 (78.5891)
Valid: 10 [ 150/390]  Loss: 1.815 (2.05)  Acc@1: 57.8125 (47.9719)  Acc@5: 82.8125 (78.7355)
Valid: 10 [ 200/390]  Loss: 1.928 (2.05)  Acc@1: 50.0000 (47.6524)  Acc@5: 76.5625 (78.6692)
Valid: 10 [ 250/390]  Loss: 2.582 (2.05)  Acc@1: 45.3125 (47.6345)  Acc@5: 76.5625 (78.5670)
Valid: 10 [ 300/390]  Loss: 2.160 (2.05)  Acc@1: 40.6250 (47.5498)  Acc@5: 76.5625 (78.4261)
Valid: 10 [ 350/390]  Loss: 1.468 (2.04)  Acc@1: 64.0625 (47.6095)  Acc@5: 84.3750 (78.5702)
Valid: 10 [ 390/390]  Loss: 2.584 (2.05)  Acc@1: 30.0000 (47.4840)  Acc@5: 67.5000 (78.5400)
valid_acc 47.484000
epoch = 10   
 genotype = Genotype(normal=[('dil_conv_5x5', 1), ('dil_conv_3x3', 0), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2902, 0.7098],
        [0.2877, 0.7123],
        [0.3018, 0.6982],
        [0.3095, 0.6905],
        [0.3116, 0.6884],
        [0.4652, 0.5348],
        [0.2505, 0.7495],
        [0.2820, 0.7180],
        [0.2628, 0.7372],
        [0.3611, 0.6389],
        [0.5371, 0.4629],
        [0.3159, 0.6841],
        [0.4904, 0.5096],
        [0.4842, 0.5158]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5487, 0.4513],
        [0.4445, 0.5555],
        [0.5301, 0.4699],
        [0.5517, 0.4483],
        [0.2685, 0.7315],
        [0.5426, 0.4574],
        [0.4130, 0.5870],
        [0.3247, 0.6753],
        [0.2927, 0.7073],
        [0.5163, 0.4837],
        [0.5442, 0.4558],
        [0.2663, 0.7337],
        [0.2960, 0.7040],
        [0.3411, 0.6589]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 11 [   0/390]  Loss: 1.472 (1.47)  Acc@1: 57.8125 (57.8125)  Acc@5: 85.9375 (85.9375)LR: 2.225e-02
Train: 11 [  50/390]  Loss: 1.639 (1.61)  Acc@1: 64.0625 (54.1667)  Acc@5: 84.3750 (84.8346)LR: 2.225e-02
Train: 11 [ 100/390]  Loss: 1.672 (1.59)  Acc@1: 54.6875 (54.7494)  Acc@5: 82.8125 (85.1485)LR: 2.225e-02
Train: 11 [ 150/390]  Loss: 1.610 (1.60)  Acc@1: 57.8125 (55.0290)  Acc@5: 82.8125 (84.7372)LR: 2.225e-02
Train: 11 [ 200/390]  Loss: 1.613 (1.61)  Acc@1: 51.5625 (55.0451)  Acc@5: 85.9375 (84.6082)LR: 2.225e-02
Train: 11 [ 250/390]  Loss: 1.423 (1.62)  Acc@1: 57.8125 (54.7124)  Acc@5: 85.9375 (84.3314)LR: 2.225e-02
Train: 11 [ 300/390]  Loss: 1.795 (1.62)  Acc@1: 46.8750 (54.5473)  Acc@5: 84.3750 (84.2919)LR: 2.225e-02
Train: 11 [ 350/390]  Loss: 1.673 (1.62)  Acc@1: 54.6875 (54.3892)  Acc@5: 79.6875 (84.2103)LR: 2.225e-02
Train: 11 [ 390/390]  Loss: 2.040 (1.62)  Acc@1: 40.0000 (54.4680)  Acc@5: 75.0000 (84.3280)LR: 2.225e-02
train_acc 54.468000
Valid: 11 [   0/390]  Loss: 2.127 (2.13)  Acc@1: 48.4375 (48.4375)  Acc@5: 78.1250 (78.1250)
Valid: 11 [  50/390]  Loss: 1.580 (1.97)  Acc@1: 54.6875 (49.5711)  Acc@5: 87.5000 (79.5650)
Valid: 11 [ 100/390]  Loss: 2.021 (1.96)  Acc@1: 48.4375 (49.5204)  Acc@5: 84.3750 (79.9814)
Valid: 11 [ 150/390]  Loss: 2.035 (1.97)  Acc@1: 54.6875 (49.1929)  Acc@5: 73.4375 (79.8324)
Valid: 11 [ 200/390]  Loss: 1.947 (1.97)  Acc@1: 51.5625 (49.1838)  Acc@5: 81.2500 (79.7886)
Valid: 11 [ 250/390]  Loss: 1.790 (1.96)  Acc@1: 51.5625 (49.3526)  Acc@5: 85.9375 (79.9614)
Valid: 11 [ 300/390]  Loss: 1.997 (1.96)  Acc@1: 48.4375 (49.1902)  Acc@5: 82.8125 (79.9834)
Valid: 11 [ 350/390]  Loss: 2.141 (1.96)  Acc@1: 39.0625 (49.1720)  Acc@5: 76.5625 (79.9457)
Valid: 11 [ 390/390]  Loss: 2.262 (1.96)  Acc@1: 37.5000 (49.2120)  Acc@5: 82.5000 (79.9600)
valid_acc 49.212000
epoch = 11   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2690, 0.7310],
        [0.2722, 0.7278],
        [0.2871, 0.7129],
        [0.2944, 0.7056],
        [0.2976, 0.7024],
        [0.4454, 0.5546],
        [0.2351, 0.7649],
        [0.2674, 0.7326],
        [0.2428, 0.7572],
        [0.3462, 0.6538],
        [0.5271, 0.4729],
        [0.2990, 0.7010],
        [0.4932, 0.5068],
        [0.4884, 0.5116]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5501, 0.4499],
        [0.4247, 0.5753],
        [0.5271, 0.4729],
        [0.5514, 0.4486],
        [0.2485, 0.7515],
        [0.5403, 0.4597],
        [0.3972, 0.6028],
        [0.3077, 0.6923],
        [0.2742, 0.7258],
        [0.5114, 0.4886],
        [0.5430, 0.4570],
        [0.2477, 0.7523],
        [0.2747, 0.7253],
        [0.3257, 0.6743]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 12 [   0/390]  Loss: 1.190 (1.19)  Acc@1: 67.1875 (67.1875)  Acc@5: 92.1875 (92.1875)LR: 2.175e-02
Train: 12 [  50/390]  Loss: 1.577 (1.48)  Acc@1: 56.2500 (57.6287)  Acc@5: 87.5000 (86.5502)LR: 2.175e-02
Train: 12 [ 100/390]  Loss: 1.580 (1.50)  Acc@1: 50.0000 (56.8843)  Acc@5: 90.6250 (86.4944)LR: 2.175e-02
Train: 12 [ 150/390]  Loss: 1.600 (1.50)  Acc@1: 53.1250 (57.2227)  Acc@5: 87.5000 (86.5170)LR: 2.175e-02
Train: 12 [ 200/390]  Loss: 1.734 (1.51)  Acc@1: 46.8750 (57.2606)  Acc@5: 84.3750 (86.5361)LR: 2.175e-02
Train: 12 [ 250/390]  Loss: 1.255 (1.52)  Acc@1: 62.5000 (56.7480)  Acc@5: 93.7500 (86.3484)LR: 2.175e-02
Train: 12 [ 300/390]  Loss: 1.700 (1.53)  Acc@1: 54.6875 (56.5978)  Acc@5: 89.0625 (86.0725)LR: 2.175e-02
Train: 12 [ 350/390]  Loss: 1.826 (1.55)  Acc@1: 56.2500 (56.3123)  Acc@5: 75.0000 (85.7906)LR: 2.175e-02
Train: 12 [ 390/390]  Loss: 1.754 (1.55)  Acc@1: 52.5000 (56.2840)  Acc@5: 82.5000 (85.6880)LR: 2.175e-02
train_acc 56.284000
Valid: 12 [   0/390]  Loss: 1.469 (1.47)  Acc@1: 62.5000 (62.5000)  Acc@5: 85.9375 (85.9375)
Valid: 12 [  50/390]  Loss: 1.956 (1.96)  Acc@1: 51.5625 (49.4179)  Acc@5: 78.1250 (79.7488)
Valid: 12 [ 100/390]  Loss: 1.749 (1.98)  Acc@1: 50.0000 (48.9480)  Acc@5: 78.1250 (79.6720)
Valid: 12 [ 150/390]  Loss: 1.847 (1.97)  Acc@1: 56.2500 (49.0894)  Acc@5: 84.3750 (80.0497)
Valid: 12 [ 200/390]  Loss: 2.166 (1.97)  Acc@1: 56.2500 (48.9039)  Acc@5: 78.1250 (79.9829)
Valid: 12 [ 250/390]  Loss: 1.908 (1.96)  Acc@1: 53.1250 (48.8670)  Acc@5: 79.6875 (80.2602)
Valid: 12 [ 300/390]  Loss: 1.942 (1.96)  Acc@1: 43.7500 (48.9255)  Acc@5: 79.6875 (80.0820)
Valid: 12 [ 350/390]  Loss: 2.059 (1.97)  Acc@1: 42.1875 (48.6690)  Acc@5: 79.6875 (79.8834)
Valid: 12 [ 390/390]  Loss: 2.197 (1.97)  Acc@1: 50.0000 (48.5920)  Acc@5: 72.5000 (79.8280)
valid_acc 48.592000
epoch = 12   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('sep_conv_3x3', 1), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2506, 0.7494],
        [0.2592, 0.7408],
        [0.2754, 0.7246],
        [0.2832, 0.7168],
        [0.2835, 0.7165],
        [0.4275, 0.5725],
        [0.2230, 0.7770],
        [0.2500, 0.7500],
        [0.2268, 0.7732],
        [0.3376, 0.6624],
        [0.5169, 0.4831],
        [0.2906, 0.7094],
        [0.4950, 0.5050],
        [0.4939, 0.5061]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5521, 0.4479],
        [0.4087, 0.5913],
        [0.5274, 0.4726],
        [0.5533, 0.4467],
        [0.2342, 0.7658],
        [0.5405, 0.4595],
        [0.3857, 0.6143],
        [0.2996, 0.7004],
        [0.2593, 0.7407],
        [0.5106, 0.4894],
        [0.5448, 0.4552],
        [0.2344, 0.7656],
        [0.2610, 0.7390],
        [0.3160, 0.6840]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 13 [   0/390]  Loss: 1.149 (1.15)  Acc@1: 71.8750 (71.8750)  Acc@5: 93.7500 (93.7500)LR: 2.121e-02
Train: 13 [  50/390]  Loss: 1.382 (1.37)  Acc@1: 54.6875 (60.6924)  Acc@5: 89.0625 (88.8787)LR: 2.121e-02
Train: 13 [ 100/390]  Loss: 1.433 (1.41)  Acc@1: 67.1875 (59.4524)  Acc@5: 85.9375 (87.9641)LR: 2.121e-02
Train: 13 [ 150/390]  Loss: 1.146 (1.45)  Acc@1: 62.5000 (58.7852)  Acc@5: 90.6250 (87.3551)LR: 2.121e-02
Train: 13 [ 200/390]  Loss: 1.531 (1.47)  Acc@1: 60.9375 (58.1779)  Acc@5: 82.8125 (86.9714)LR: 2.121e-02
Train: 13 [ 250/390]  Loss: 1.970 (1.47)  Acc@1: 46.8750 (58.0304)  Acc@5: 78.1250 (87.0518)LR: 2.121e-02
Train: 13 [ 300/390]  Loss: 1.709 (1.48)  Acc@1: 51.5625 (57.7606)  Acc@5: 79.6875 (86.8719)LR: 2.121e-02
Train: 13 [ 350/390]  Loss: 1.676 (1.49)  Acc@1: 56.2500 (57.5009)  Acc@5: 81.2500 (86.7121)LR: 2.121e-02
Train: 13 [ 390/390]  Loss: 1.289 (1.50)  Acc@1: 57.5000 (57.4120)  Acc@5: 95.0000 (86.6880)LR: 2.121e-02
train_acc 57.412000
Valid: 13 [   0/390]  Loss: 2.252 (2.25)  Acc@1: 46.8750 (46.8750)  Acc@5: 75.0000 (75.0000)
Valid: 13 [  50/390]  Loss: 1.792 (1.93)  Acc@1: 43.7500 (50.5821)  Acc@5: 81.2500 (81.3725)
Valid: 13 [ 100/390]  Loss: 1.962 (1.93)  Acc@1: 43.7500 (49.8917)  Acc@5: 79.6875 (80.8323)
Valid: 13 [ 150/390]  Loss: 2.232 (1.93)  Acc@1: 35.9375 (50.2483)  Acc@5: 73.4375 (80.4222)
Valid: 13 [ 200/390]  Loss: 1.945 (1.93)  Acc@1: 50.0000 (50.6297)  Acc@5: 81.2500 (80.2239)
Valid: 13 [ 250/390]  Loss: 2.125 (1.95)  Acc@1: 42.1875 (50.3548)  Acc@5: 71.8750 (79.8867)
Valid: 13 [ 300/390]  Loss: 2.115 (1.94)  Acc@1: 45.3125 (50.3063)  Acc@5: 79.6875 (79.8484)
Valid: 13 [ 350/390]  Loss: 2.354 (1.93)  Acc@1: 42.1875 (50.4051)  Acc@5: 73.4375 (79.8656)
Valid: 13 [ 390/390]  Loss: 2.168 (1.93)  Acc@1: 42.5000 (50.4280)  Acc@5: 75.0000 (79.8960)
valid_acc 50.428000
epoch = 13   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_5x5', 0), ('dil_conv_3x3', 2), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2370, 0.7630],
        [0.2466, 0.7534],
        [0.2665, 0.7335],
        [0.2690, 0.7310],
        [0.2668, 0.7332],
        [0.4091, 0.5909],
        [0.2126, 0.7874],
        [0.2331, 0.7669],
        [0.2077, 0.7923],
        [0.3221, 0.6779],
        [0.5023, 0.4977],
        [0.2751, 0.7249],
        [0.4975, 0.5025],
        [0.5016, 0.4984]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5487, 0.4513],
        [0.3952, 0.6048],
        [0.5254, 0.4746],
        [0.5538, 0.4462],
        [0.2212, 0.7788],
        [0.5401, 0.4599],
        [0.3763, 0.6237],
        [0.2873, 0.7127],
        [0.2428, 0.7572],
        [0.5091, 0.4909],
        [0.5480, 0.4520],
        [0.2180, 0.7820],
        [0.2441, 0.7559],
        [0.3015, 0.6985]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 14 [   0/390]  Loss: 1.618 (1.62)  Acc@1: 57.8125 (57.8125)  Acc@5: 79.6875 (79.6875)LR: 2.065e-02
Train: 14 [  50/390]  Loss: 1.399 (1.35)  Acc@1: 60.9375 (61.4890)  Acc@5: 82.8125 (88.0515)LR: 2.065e-02
Train: 14 [ 100/390]  Loss: 1.538 (1.37)  Acc@1: 57.8125 (60.5507)  Acc@5: 87.5000 (88.0879)LR: 2.065e-02
Train: 14 [ 150/390]  Loss: 1.578 (1.39)  Acc@1: 54.6875 (60.3063)  Acc@5: 81.2500 (87.8829)LR: 2.065e-02
Train: 14 [ 200/390]  Loss: 1.459 (1.40)  Acc@1: 56.2500 (59.8103)  Acc@5: 90.6250 (87.9975)LR: 2.065e-02
Train: 14 [ 250/390]  Loss: 1.481 (1.42)  Acc@1: 56.2500 (59.3750)  Acc@5: 87.5000 (87.7117)LR: 2.065e-02
Train: 14 [ 300/390]  Loss: 1.344 (1.42)  Acc@1: 57.8125 (59.2556)  Acc@5: 87.5000 (87.5571)LR: 2.065e-02
Train: 14 [ 350/390]  Loss: 1.353 (1.43)  Acc@1: 57.8125 (59.2236)  Acc@5: 93.7500 (87.6068)LR: 2.065e-02
Train: 14 [ 390/390]  Loss: 1.343 (1.43)  Acc@1: 55.0000 (59.1320)  Acc@5: 92.5000 (87.5800)LR: 2.065e-02
train_acc 59.132000
Valid: 14 [   0/390]  Loss: 1.810 (1.81)  Acc@1: 53.1250 (53.1250)  Acc@5: 84.3750 (84.3750)
Valid: 14 [  50/390]  Loss: 1.719 (1.86)  Acc@1: 59.3750 (52.8186)  Acc@5: 81.2500 (81.5257)
Valid: 14 [ 100/390]  Loss: 1.611 (1.87)  Acc@1: 53.1250 (52.7382)  Acc@5: 84.3750 (81.4511)
Valid: 14 [ 150/390]  Loss: 2.273 (1.88)  Acc@1: 51.5625 (52.2661)  Acc@5: 75.0000 (81.5604)
Valid: 14 [ 200/390]  Loss: 2.170 (1.88)  Acc@1: 45.3125 (52.0522)  Acc@5: 78.1250 (81.6698)
Valid: 14 [ 250/390]  Loss: 2.326 (1.86)  Acc@1: 45.3125 (52.0854)  Acc@5: 76.5625 (81.8850)
Valid: 14 [ 300/390]  Loss: 2.086 (1.84)  Acc@1: 43.7500 (52.3360)  Acc@5: 73.4375 (82.0702)
Valid: 14 [ 350/390]  Loss: 1.538 (1.84)  Acc@1: 56.2500 (52.2391)  Acc@5: 87.5000 (82.0468)
Valid: 14 [ 390/390]  Loss: 2.516 (1.86)  Acc@1: 45.0000 (52.0240)  Acc@5: 75.0000 (81.8800)
valid_acc 52.024000
epoch = 14   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('dil_conv_5x5', 0), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2227, 0.7773],
        [0.2333, 0.7667],
        [0.2578, 0.7422],
        [0.2605, 0.7395],
        [0.2564, 0.7436],
        [0.3909, 0.6091],
        [0.2063, 0.7937],
        [0.2238, 0.7762],
        [0.1926, 0.8074],
        [0.3113, 0.6887],
        [0.4852, 0.5148],
        [0.2681, 0.7319],
        [0.4984, 0.5016],
        [0.5101, 0.4899]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5461, 0.4539],
        [0.3824, 0.6176],
        [0.5221, 0.4779],
        [0.5559, 0.4441],
        [0.2086, 0.7914],
        [0.5358, 0.4642],
        [0.3665, 0.6335],
        [0.2730, 0.7270],
        [0.2281, 0.7719],
        [0.5046, 0.4954],
        [0.5490, 0.4510],
        [0.2050, 0.7950],
        [0.2278, 0.7722],
        [0.2946, 0.7054]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 15 [   0/390]  Loss: 1.141 (1.14)  Acc@1: 65.6250 (65.6250)  Acc@5: 93.7500 (93.7500)LR: 2.005e-02
Train: 15 [  50/390]  Loss: 1.468 (1.34)  Acc@1: 64.0625 (61.9485)  Acc@5: 85.9375 (89.0931)LR: 2.005e-02
Train: 15 [ 100/390]  Loss: 1.167 (1.36)  Acc@1: 62.5000 (60.9684)  Acc@5: 90.6250 (88.6603)LR: 2.005e-02
Train: 15 [ 150/390]  Loss: 1.481 (1.37)  Acc@1: 56.2500 (60.5753)  Acc@5: 87.5000 (88.5451)LR: 2.005e-02
Train: 15 [ 200/390]  Loss: 1.345 (1.37)  Acc@1: 60.9375 (60.7587)  Acc@5: 84.3750 (88.5028)LR: 2.005e-02
Train: 15 [ 250/390]  Loss: 1.404 (1.38)  Acc@1: 60.9375 (60.4955)  Acc@5: 84.3750 (88.3902)LR: 2.005e-02
Train: 15 [ 300/390]  Loss: 1.208 (1.38)  Acc@1: 64.0625 (60.4184)  Acc@5: 90.6250 (88.3617)LR: 2.005e-02
Train: 15 [ 350/390]  Loss: 1.266 (1.38)  Acc@1: 70.3125 (60.4167)  Acc@5: 87.5000 (88.2612)LR: 2.005e-02
Train: 15 [ 390/390]  Loss: 1.433 (1.39)  Acc@1: 55.0000 (60.0760)  Acc@5: 87.5000 (88.2440)LR: 2.005e-02
train_acc 60.076000
Valid: 15 [   0/390]  Loss: 1.817 (1.82)  Acc@1: 50.0000 (50.0000)  Acc@5: 82.8125 (82.8125)
Valid: 15 [  50/390]  Loss: 2.186 (1.94)  Acc@1: 43.7500 (50.8578)  Acc@5: 76.5625 (80.2390)
Valid: 15 [ 100/390]  Loss: 2.098 (1.92)  Acc@1: 50.0000 (50.9282)  Acc@5: 79.6875 (81.0025)
Valid: 15 [ 150/390]  Loss: 2.402 (1.93)  Acc@1: 50.0000 (50.8485)  Acc@5: 65.6250 (80.7533)
Valid: 15 [ 200/390]  Loss: 1.780 (1.92)  Acc@1: 51.5625 (50.8162)  Acc@5: 82.8125 (80.7447)
Valid: 15 [ 250/390]  Loss: 2.049 (1.94)  Acc@1: 48.4375 (50.6848)  Acc@5: 79.6875 (80.5528)
Valid: 15 [ 300/390]  Loss: 2.133 (1.95)  Acc@1: 35.9375 (50.6385)  Acc@5: 76.5625 (80.5752)
Valid: 15 [ 350/390]  Loss: 2.260 (1.96)  Acc@1: 43.7500 (50.5164)  Acc@5: 78.1250 (80.5556)
Valid: 15 [ 390/390]  Loss: 1.756 (1.96)  Acc@1: 50.0000 (50.3160)  Acc@5: 85.0000 (80.5280)
valid_acc 50.316000
epoch = 15   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.2097, 0.7903],
        [0.2227, 0.7773],
        [0.2517, 0.7483],
        [0.2462, 0.7538],
        [0.2425, 0.7575],
        [0.3764, 0.6236],
        [0.1968, 0.8032],
        [0.2125, 0.7875],
        [0.1775, 0.8225],
        [0.3033, 0.6967],
        [0.4757, 0.5243],
        [0.2593, 0.7407],
        [0.5022, 0.4978],
        [0.5140, 0.4860]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5382, 0.4618],
        [0.3679, 0.6321],
        [0.5137, 0.4863],
        [0.5553, 0.4447],
        [0.1947, 0.8053],
        [0.5298, 0.4702],
        [0.3526, 0.6474],
        [0.2616, 0.7384],
        [0.2129, 0.7871],
        [0.4951, 0.5049],
        [0.5488, 0.4512],
        [0.1914, 0.8086],
        [0.2130, 0.7870],
        [0.2837, 0.7163]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 16 [   0/390]  Loss: 1.174 (1.17)  Acc@1: 68.7500 (68.7500)  Acc@5: 89.0625 (89.0625)LR: 1.943e-02
Train: 16 [  50/390]  Loss: 1.314 (1.32)  Acc@1: 65.6250 (62.0098)  Acc@5: 84.3750 (89.1544)LR: 1.943e-02
Train: 16 [ 100/390]  Loss: 1.211 (1.32)  Acc@1: 71.8750 (62.2989)  Acc@5: 92.1875 (89.3719)LR: 1.943e-02
Train: 16 [ 150/390]  Loss: 1.151 (1.32)  Acc@1: 70.3125 (62.5310)  Acc@5: 93.7500 (89.3729)LR: 1.943e-02
Train: 16 [ 200/390]  Loss: 1.382 (1.32)  Acc@1: 64.0625 (62.2201)  Acc@5: 89.0625 (89.3113)LR: 1.943e-02
Train: 16 [ 250/390]  Loss: 1.174 (1.33)  Acc@1: 67.1875 (61.8650)  Acc@5: 92.1875 (89.1995)LR: 1.943e-02
Train: 16 [ 300/390]  Loss: 1.272 (1.32)  Acc@1: 64.0625 (61.9601)  Acc@5: 92.1875 (89.1507)LR: 1.943e-02
Train: 16 [ 350/390]  Loss: 1.307 (1.33)  Acc@1: 59.3750 (61.8056)  Acc@5: 85.9375 (89.1471)LR: 1.943e-02
Train: 16 [ 390/390]  Loss: 1.365 (1.33)  Acc@1: 65.0000 (61.8840)  Acc@5: 90.0000 (89.1520)LR: 1.943e-02
train_acc 61.884000
Valid: 16 [   0/390]  Loss: 2.047 (2.05)  Acc@1: 48.4375 (48.4375)  Acc@5: 79.6875 (79.6875)
Valid: 16 [  50/390]  Loss: 2.094 (1.87)  Acc@1: 42.1875 (51.0723)  Acc@5: 76.5625 (81.9547)
Valid: 16 [ 100/390]  Loss: 2.232 (1.90)  Acc@1: 45.3125 (50.9437)  Acc@5: 78.1250 (81.2191)
Valid: 16 [ 150/390]  Loss: 1.829 (1.91)  Acc@1: 53.1250 (50.8899)  Acc@5: 84.3750 (81.1155)
Valid: 16 [ 200/390]  Loss: 1.981 (1.93)  Acc@1: 50.0000 (50.6219)  Acc@5: 75.0000 (80.8458)
Valid: 16 [ 250/390]  Loss: 2.016 (1.93)  Acc@1: 48.4375 (50.5727)  Acc@5: 81.2500 (80.9014)
Valid: 16 [ 300/390]  Loss: 1.670 (1.93)  Acc@1: 48.4375 (50.6022)  Acc@5: 89.0625 (81.0475)
Valid: 16 [ 350/390]  Loss: 2.098 (1.93)  Acc@1: 40.6250 (50.6054)  Acc@5: 75.0000 (81.0897)
Valid: 16 [ 390/390]  Loss: 1.932 (1.93)  Acc@1: 45.0000 (50.4240)  Acc@5: 70.0000 (81.1160)
valid_acc 50.424000
epoch = 16   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1966, 0.8034],
        [0.2149, 0.7851],
        [0.2454, 0.7546],
        [0.2375, 0.7625],
        [0.2319, 0.7681],
        [0.3634, 0.6366],
        [0.1916, 0.8084],
        [0.2021, 0.7979],
        [0.1650, 0.8350],
        [0.2925, 0.7075],
        [0.4597, 0.5403],
        [0.2569, 0.7431],
        [0.4974, 0.5026],
        [0.5186, 0.4814]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5334, 0.4666],
        [0.3573, 0.6427],
        [0.5066, 0.4934],
        [0.5541, 0.4459],
        [0.1825, 0.8175],
        [0.5250, 0.4750],
        [0.3363, 0.6637],
        [0.2511, 0.7489],
        [0.2010, 0.7990],
        [0.4887, 0.5113],
        [0.5464, 0.4536],
        [0.1793, 0.8207],
        [0.1995, 0.8005],
        [0.2753, 0.7247]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 17 [   0/390]  Loss: 1.520 (1.52)  Acc@1: 57.8125 (57.8125)  Acc@5: 85.9375 (85.9375)LR: 1.878e-02
Train: 17 [  50/390]  Loss: 1.162 (1.24)  Acc@1: 56.2500 (63.5723)  Acc@5: 92.1875 (90.6250)LR: 1.878e-02
Train: 17 [ 100/390]  Loss: 0.8734 (1.27)  Acc@1: 75.0000 (63.1498)  Acc@5: 92.1875 (89.8824)LR: 1.878e-02
Train: 17 [ 150/390]  Loss: 1.193 (1.26)  Acc@1: 70.3125 (63.3175)  Acc@5: 89.0625 (90.1076)LR: 1.878e-02
Train: 17 [ 200/390]  Loss: 1.443 (1.27)  Acc@1: 64.0625 (62.9042)  Acc@5: 85.9375 (90.0886)LR: 1.878e-02
Train: 17 [ 250/390]  Loss: 1.646 (1.28)  Acc@1: 50.0000 (62.5934)  Acc@5: 84.3750 (89.8780)LR: 1.878e-02
Train: 17 [ 300/390]  Loss: 1.186 (1.29)  Acc@1: 70.3125 (62.4169)  Acc@5: 89.0625 (89.5920)LR: 1.878e-02
Train: 17 [ 350/390]  Loss: 1.347 (1.29)  Acc@1: 62.5000 (62.3397)  Acc@5: 87.5000 (89.5433)LR: 1.878e-02
Train: 17 [ 390/390]  Loss: 1.725 (1.30)  Acc@1: 50.0000 (62.1880)  Acc@5: 82.5000 (89.5040)LR: 1.878e-02
train_acc 62.188000
Valid: 17 [   0/390]  Loss: 1.608 (1.61)  Acc@1: 57.8125 (57.8125)  Acc@5: 82.8125 (82.8125)
Valid: 17 [  50/390]  Loss: 1.661 (1.77)  Acc@1: 54.6875 (53.3395)  Acc@5: 87.5000 (83.0270)
Valid: 17 [ 100/390]  Loss: 1.825 (1.77)  Acc@1: 53.1250 (53.2178)  Acc@5: 78.1250 (82.6423)
Valid: 17 [ 150/390]  Loss: 1.602 (1.78)  Acc@1: 54.6875 (53.8493)  Acc@5: 87.5000 (82.2951)
Valid: 17 [ 200/390]  Loss: 1.958 (1.78)  Acc@1: 54.6875 (53.7702)  Acc@5: 79.6875 (82.3616)
Valid: 17 [ 250/390]  Loss: 1.899 (1.77)  Acc@1: 53.1250 (53.8969)  Acc@5: 82.8125 (82.4888)
Valid: 17 [ 300/390]  Loss: 1.402 (1.77)  Acc@1: 59.3750 (53.7375)  Acc@5: 84.3750 (82.5322)
Valid: 17 [ 350/390]  Loss: 1.495 (1.76)  Acc@1: 56.2500 (54.0821)  Acc@5: 89.0625 (82.8036)
Valid: 17 [ 390/390]  Loss: 2.318 (1.76)  Acc@1: 47.5000 (54.0680)  Acc@5: 75.0000 (82.8200)
valid_acc 54.068000
epoch = 17   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1850, 0.8150],
        [0.2059, 0.7941],
        [0.2410, 0.7590],
        [0.2260, 0.7740],
        [0.2207, 0.7793],
        [0.3474, 0.6526],
        [0.1864, 0.8136],
        [0.1955, 0.8045],
        [0.1552, 0.8448],
        [0.2854, 0.7146],
        [0.4395, 0.5605],
        [0.2488, 0.7512],
        [0.4920, 0.5080],
        [0.5218, 0.4782]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5325, 0.4675],
        [0.3467, 0.6533],
        [0.5031, 0.4969],
        [0.5534, 0.4466],
        [0.1715, 0.8285],
        [0.5228, 0.4772],
        [0.3215, 0.6785],
        [0.2404, 0.7596],
        [0.1898, 0.8102],
        [0.4829, 0.5171],
        [0.5437, 0.4563],
        [0.1676, 0.8324],
        [0.1875, 0.8125],
        [0.2696, 0.7304]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 18 [   0/390]  Loss: 1.439 (1.44)  Acc@1: 54.6875 (54.6875)  Acc@5: 92.1875 (92.1875)LR: 1.811e-02
Train: 18 [  50/390]  Loss: 1.435 (1.18)  Acc@1: 62.5000 (65.1042)  Acc@5: 84.3750 (91.8505)LR: 1.811e-02
Train: 18 [ 100/390]  Loss: 1.249 (1.20)  Acc@1: 64.0625 (65.2073)  Acc@5: 95.3125 (91.3212)LR: 1.811e-02
Train: 18 [ 150/390]  Loss: 1.423 (1.21)  Acc@1: 64.0625 (64.8903)  Acc@5: 87.5000 (91.1010)LR: 1.811e-02
Train: 18 [ 200/390]  Loss: 1.480 (1.22)  Acc@1: 53.1250 (64.5833)  Acc@5: 89.0625 (90.7572)LR: 1.811e-02
Train: 18 [ 250/390]  Loss: 1.247 (1.23)  Acc@1: 67.1875 (64.3924)  Acc@5: 92.1875 (90.5690)LR: 1.811e-02
Train: 18 [ 300/390]  Loss: 1.250 (1.24)  Acc@1: 67.1875 (64.1715)  Acc@5: 85.9375 (90.4381)LR: 1.811e-02
Train: 18 [ 350/390]  Loss: 1.755 (1.25)  Acc@1: 53.1250 (64.0491)  Acc@5: 81.2500 (90.2199)LR: 1.811e-02
Train: 18 [ 390/390]  Loss: 1.782 (1.26)  Acc@1: 35.0000 (63.7800)  Acc@5: 87.5000 (90.0960)LR: 1.811e-02
train_acc 63.780000
Valid: 18 [   0/390]  Loss: 1.291 (1.29)  Acc@1: 59.3750 (59.3750)  Acc@5: 92.1875 (92.1875)
Valid: 18 [  50/390]  Loss: 1.825 (1.79)  Acc@1: 53.1250 (54.2586)  Acc@5: 81.2500 (83.1801)
Valid: 18 [ 100/390]  Loss: 1.822 (1.81)  Acc@1: 53.1250 (53.6046)  Acc@5: 82.8125 (82.8899)
Valid: 18 [ 150/390]  Loss: 1.842 (1.79)  Acc@1: 54.6875 (53.5596)  Acc@5: 87.5000 (83.0712)
Valid: 18 [ 200/390]  Loss: 1.908 (1.80)  Acc@1: 45.3125 (53.3504)  Acc@5: 82.8125 (82.8358)
Valid: 18 [ 250/390]  Loss: 1.841 (1.80)  Acc@1: 54.6875 (53.4923)  Acc@5: 82.8125 (82.9370)
Valid: 18 [ 300/390]  Loss: 1.939 (1.80)  Acc@1: 50.0000 (53.4624)  Acc@5: 81.2500 (82.7191)
Valid: 18 [ 350/390]  Loss: 1.840 (1.81)  Acc@1: 50.0000 (53.5034)  Acc@5: 85.9375 (82.6923)
Valid: 18 [ 390/390]  Loss: 1.604 (1.81)  Acc@1: 52.5000 (53.3800)  Acc@5: 82.5000 (82.6760)
valid_acc 53.380000
epoch = 18   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1732, 0.8268],
        [0.1959, 0.8041],
        [0.2347, 0.7653],
        [0.2165, 0.7835],
        [0.2122, 0.7878],
        [0.3331, 0.6669],
        [0.1813, 0.8187],
        [0.1856, 0.8144],
        [0.1446, 0.8554],
        [0.2831, 0.7169],
        [0.4234, 0.5766],
        [0.2433, 0.7567],
        [0.4907, 0.5093],
        [0.5275, 0.4725]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5308, 0.4692],
        [0.3350, 0.6650],
        [0.5006, 0.4994],
        [0.5546, 0.4454],
        [0.1623, 0.8377],
        [0.5200, 0.4800],
        [0.3077, 0.6923],
        [0.2288, 0.7712],
        [0.1780, 0.8220],
        [0.4799, 0.5201],
        [0.5444, 0.4556],
        [0.1566, 0.8434],
        [0.1743, 0.8257],
        [0.2599, 0.7401]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 19 [   0/390]  Loss: 0.9962 (0.996)  Acc@1: 70.3125 (70.3125)  Acc@5: 93.7500 (93.7500)LR: 1.742e-02
Train: 19 [  50/390]  Loss: 1.064 (1.09)  Acc@1: 65.6250 (67.6164)  Acc@5: 90.6250 (92.9534)LR: 1.742e-02
Train: 19 [ 100/390]  Loss: 1.144 (1.12)  Acc@1: 65.6250 (66.9554)  Acc@5: 87.5000 (92.2803)LR: 1.742e-02
Train: 19 [ 150/390]  Loss: 1.260 (1.15)  Acc@1: 70.3125 (66.3907)  Acc@5: 90.6250 (91.9805)LR: 1.742e-02
Train: 19 [ 200/390]  Loss: 1.477 (1.15)  Acc@1: 59.3750 (65.9904)  Acc@5: 82.8125 (91.6356)LR: 1.742e-02
Train: 19 [ 250/390]  Loss: 1.446 (1.17)  Acc@1: 54.6875 (65.7308)  Acc@5: 85.9375 (91.3720)LR: 1.742e-02
Train: 19 [ 300/390]  Loss: 1.185 (1.18)  Acc@1: 65.6250 (65.5212)  Acc@5: 87.5000 (91.0039)LR: 1.742e-02
Train: 19 [ 350/390]  Loss: 1.298 (1.19)  Acc@1: 67.1875 (65.3134)  Acc@5: 90.6250 (90.9054)LR: 1.742e-02
Train: 19 [ 390/390]  Loss: 1.358 (1.20)  Acc@1: 57.5000 (65.0400)  Acc@5: 90.0000 (90.7600)LR: 1.742e-02
train_acc 65.040000
Valid: 19 [   0/390]  Loss: 1.829 (1.83)  Acc@1: 54.6875 (54.6875)  Acc@5: 82.8125 (82.8125)
Valid: 19 [  50/390]  Loss: 1.562 (1.86)  Acc@1: 59.3750 (53.1556)  Acc@5: 81.2500 (81.2806)
Valid: 19 [ 100/390]  Loss: 1.543 (1.84)  Acc@1: 64.0625 (53.3725)  Acc@5: 82.8125 (81.7296)
Valid: 19 [ 150/390]  Loss: 1.949 (1.83)  Acc@1: 50.0000 (53.5493)  Acc@5: 81.2500 (81.7881)
Valid: 19 [ 200/390]  Loss: 1.817 (1.83)  Acc@1: 60.9375 (53.6458)  Acc@5: 76.5625 (82.0896)
Valid: 19 [ 250/390]  Loss: 2.102 (1.81)  Acc@1: 42.1875 (53.9778)  Acc@5: 76.5625 (82.4203)
Valid: 19 [ 300/390]  Loss: 1.374 (1.81)  Acc@1: 57.8125 (53.9192)  Acc@5: 89.0625 (82.3765)
Valid: 19 [ 350/390]  Loss: 2.146 (1.81)  Acc@1: 45.3125 (53.9485)  Acc@5: 79.6875 (82.4475)
Valid: 19 [ 390/390]  Loss: 1.384 (1.81)  Acc@1: 67.5000 (53.9200)  Acc@5: 87.5000 (82.5080)
valid_acc 53.920000
epoch = 19   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1629, 0.8371],
        [0.1891, 0.8109],
        [0.2324, 0.7676],
        [0.2088, 0.7912],
        [0.2036, 0.7964],
        [0.3193, 0.6807],
        [0.1745, 0.8255],
        [0.1781, 0.8219],
        [0.1336, 0.8664],
        [0.2772, 0.7228],
        [0.4042, 0.5958],
        [0.2359, 0.7641],
        [0.4949, 0.5051],
        [0.5278, 0.4722]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5287, 0.4713],
        [0.3250, 0.6750],
        [0.4977, 0.5023],
        [0.5515, 0.4485],
        [0.1553, 0.8447],
        [0.5160, 0.4840],
        [0.2936, 0.7064],
        [0.2217, 0.7783],
        [0.1681, 0.8319],
        [0.4743, 0.5257],
        [0.5407, 0.4593],
        [0.1498, 0.8502],
        [0.1663, 0.8337],
        [0.2545, 0.7455]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 20 [   0/390]  Loss: 1.304 (1.30)  Acc@1: 67.1875 (67.1875)  Acc@5: 85.9375 (85.9375)LR: 1.671e-02
Train: 20 [  50/390]  Loss: 0.9990 (1.10)  Acc@1: 67.1875 (68.1373)  Acc@5: 98.4375 (91.9118)LR: 1.671e-02
Train: 20 [ 100/390]  Loss: 1.015 (1.09)  Acc@1: 73.4375 (67.9920)  Acc@5: 92.1875 (92.3113)LR: 1.671e-02
Train: 20 [ 150/390]  Loss: 1.161 (1.10)  Acc@1: 64.0625 (67.9739)  Acc@5: 90.6250 (92.3324)LR: 1.671e-02
Train: 20 [ 200/390]  Loss: 1.031 (1.12)  Acc@1: 70.3125 (67.2886)  Acc@5: 95.3125 (92.0320)LR: 1.671e-02
Train: 20 [ 250/390]  Loss: 1.057 (1.12)  Acc@1: 68.7500 (67.1501)  Acc@5: 93.7500 (92.0132)LR: 1.671e-02
Train: 20 [ 300/390]  Loss: 1.450 (1.14)  Acc@1: 56.2500 (66.4296)  Acc@5: 82.8125 (91.7307)LR: 1.671e-02
Train: 20 [ 350/390]  Loss: 0.9586 (1.15)  Acc@1: 75.0000 (66.1993)  Acc@5: 95.3125 (91.5821)LR: 1.671e-02
Train: 20 [ 390/390]  Loss: 1.243 (1.15)  Acc@1: 62.5000 (66.0760)  Acc@5: 92.5000 (91.4680)LR: 1.671e-02
train_acc 66.076000
Valid: 20 [   0/390]  Loss: 1.924 (1.92)  Acc@1: 56.2500 (56.2500)  Acc@5: 78.1250 (78.1250)
Valid: 20 [  50/390]  Loss: 1.919 (1.89)  Acc@1: 53.1250 (53.0025)  Acc@5: 82.8125 (81.5257)
Valid: 20 [ 100/390]  Loss: 1.580 (1.87)  Acc@1: 50.0000 (52.5526)  Acc@5: 84.3750 (82.1627)
Valid: 20 [ 150/390]  Loss: 2.391 (1.86)  Acc@1: 48.4375 (52.8353)  Acc@5: 75.0000 (81.9847)
Valid: 20 [ 200/390]  Loss: 1.830 (1.87)  Acc@1: 45.3125 (52.7130)  Acc@5: 82.8125 (81.9885)
Valid: 20 [ 250/390]  Loss: 2.145 (1.86)  Acc@1: 45.3125 (52.7764)  Acc@5: 78.1250 (82.3020)
Valid: 20 [ 300/390]  Loss: 2.391 (1.86)  Acc@1: 40.6250 (52.6578)  Acc@5: 79.6875 (82.3297)
Valid: 20 [ 350/390]  Loss: 2.135 (1.86)  Acc@1: 48.4375 (52.6932)  Acc@5: 84.3750 (82.3050)
Valid: 20 [ 390/390]  Loss: 1.242 (1.87)  Acc@1: 67.5000 (52.5240)  Acc@5: 87.5000 (82.2720)
valid_acc 52.524000
epoch = 20   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1532, 0.8468],
        [0.1805, 0.8195],
        [0.2273, 0.7727],
        [0.2006, 0.7994],
        [0.1938, 0.8062],
        [0.3052, 0.6948],
        [0.1706, 0.8294],
        [0.1693, 0.8307],
        [0.1238, 0.8762],
        [0.2687, 0.7313],
        [0.3901, 0.6099],
        [0.2287, 0.7713],
        [0.4908, 0.5092],
        [0.5283, 0.4717]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5225, 0.4775],
        [0.3080, 0.6920],
        [0.4928, 0.5072],
        [0.5531, 0.4469],
        [0.1485, 0.8515],
        [0.5088, 0.4912],
        [0.2816, 0.7184],
        [0.2169, 0.7831],
        [0.1578, 0.8422],
        [0.4652, 0.5348],
        [0.5418, 0.4582],
        [0.1409, 0.8591],
        [0.1541, 0.8459],
        [0.2461, 0.7539]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 21 [   0/390]  Loss: 1.266 (1.27)  Acc@1: 67.1875 (67.1875)  Acc@5: 93.7500 (93.7500)LR: 1.598e-02
Train: 21 [  50/390]  Loss: 1.364 (1.04)  Acc@1: 64.0625 (68.8725)  Acc@5: 87.5000 (93.3211)LR: 1.598e-02
Train: 21 [ 100/390]  Loss: 0.9384 (1.05)  Acc@1: 67.1875 (68.6726)  Acc@5: 96.8750 (93.1312)LR: 1.598e-02
Train: 21 [ 150/390]  Loss: 0.9041 (1.05)  Acc@1: 75.0000 (68.7500)  Acc@5: 93.7500 (92.9636)LR: 1.598e-02
Train: 21 [ 200/390]  Loss: 0.9580 (1.07)  Acc@1: 70.3125 (68.4235)  Acc@5: 100.0000 (92.8405)LR: 1.598e-02
Train: 21 [ 250/390]  Loss: 1.039 (1.07)  Acc@1: 75.0000 (68.6068)  Acc@5: 87.5000 (92.7540)LR: 1.598e-02
Train: 21 [ 300/390]  Loss: 1.273 (1.09)  Acc@1: 60.9375 (67.9817)  Acc@5: 87.5000 (92.3277)LR: 1.598e-02
Train: 21 [ 350/390]  Loss: 1.092 (1.10)  Acc@1: 68.7500 (67.8241)  Acc@5: 95.3125 (92.1741)LR: 1.598e-02
Train: 21 [ 390/390]  Loss: 1.309 (1.10)  Acc@1: 70.0000 (67.6520)  Acc@5: 92.5000 (92.1320)LR: 1.598e-02
train_acc 67.652000
Valid: 21 [   0/390]  Loss: 1.742 (1.74)  Acc@1: 56.2500 (56.2500)  Acc@5: 81.2500 (81.2500)
Valid: 21 [  50/390]  Loss: 1.821 (1.72)  Acc@1: 48.4375 (54.5343)  Acc@5: 89.0625 (83.7316)
Valid: 21 [ 100/390]  Loss: 1.499 (1.76)  Acc@1: 64.0625 (54.5328)  Acc@5: 87.5000 (83.3385)
Valid: 21 [ 150/390]  Loss: 1.998 (1.75)  Acc@1: 42.1875 (55.0807)  Acc@5: 81.2500 (83.6714)
Valid: 21 [ 200/390]  Loss: 1.718 (1.74)  Acc@1: 62.5000 (55.1306)  Acc@5: 79.6875 (83.7453)
Valid: 21 [ 250/390]  Loss: 1.711 (1.73)  Acc@1: 54.6875 (55.3598)  Acc@5: 87.5000 (83.7276)
Valid: 21 [ 300/390]  Loss: 1.627 (1.74)  Acc@1: 57.8125 (55.2170)  Acc@5: 87.5000 (83.8715)
Valid: 21 [ 350/390]  Loss: 1.815 (1.74)  Acc@1: 60.9375 (55.1549)  Acc@5: 84.3750 (83.8631)
Valid: 21 [ 390/390]  Loss: 1.545 (1.74)  Acc@1: 42.5000 (55.0640)  Acc@5: 85.0000 (83.8440)
valid_acc 55.064000
epoch = 21   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1440, 0.8560],
        [0.1754, 0.8246],
        [0.2244, 0.7756],
        [0.1957, 0.8043],
        [0.1876, 0.8124],
        [0.2870, 0.7130],
        [0.1678, 0.8322],
        [0.1649, 0.8351],
        [0.1163, 0.8837],
        [0.2642, 0.7358],
        [0.3734, 0.6266],
        [0.2262, 0.7738],
        [0.4927, 0.5073],
        [0.5290, 0.4710]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5170, 0.4830],
        [0.3006, 0.6994],
        [0.4849, 0.5151],
        [0.5521, 0.4479],
        [0.1400, 0.8600],
        [0.5000, 0.5000],
        [0.2713, 0.7287],
        [0.2086, 0.7914],
        [0.1502, 0.8498],
        [0.4554, 0.5446],
        [0.5408, 0.4592],
        [0.1319, 0.8681],
        [0.1456, 0.8544],
        [0.2364, 0.7636]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 22 [   0/390]  Loss: 0.6258 (0.626)  Acc@1: 81.2500 (81.2500)  Acc@5: 98.4375 (98.4375)LR: 1.525e-02
Train: 22 [  50/390]  Loss: 0.9871 (1.00)  Acc@1: 76.5625 (71.3848)  Acc@5: 92.1875 (93.1066)LR: 1.525e-02
Train: 22 [ 100/390]  Loss: 1.037 (1.00)  Acc@1: 68.7500 (71.2098)  Acc@5: 93.7500 (93.0848)LR: 1.525e-02
Train: 22 [ 150/390]  Loss: 1.477 (1.03)  Acc@1: 60.9375 (70.3125)  Acc@5: 81.2500 (92.7359)LR: 1.525e-02
Train: 22 [ 200/390]  Loss: 0.7710 (1.04)  Acc@1: 81.2500 (69.6984)  Acc@5: 93.7500 (92.7705)LR: 1.525e-02
Train: 22 [ 250/390]  Loss: 1.041 (1.05)  Acc@1: 68.7500 (69.0426)  Acc@5: 93.7500 (92.7602)LR: 1.525e-02
Train: 22 [ 300/390]  Loss: 0.9979 (1.05)  Acc@1: 71.8750 (68.9161)  Acc@5: 95.3125 (92.7377)LR: 1.525e-02
Train: 22 [ 350/390]  Loss: 1.254 (1.06)  Acc@1: 60.9375 (68.5986)  Acc@5: 90.6250 (92.6460)LR: 1.525e-02
Train: 22 [ 390/390]  Loss: 1.256 (1.07)  Acc@1: 62.5000 (68.3680)  Acc@5: 92.5000 (92.6040)LR: 1.525e-02
train_acc 68.368000
Valid: 22 [   0/390]  Loss: 1.772 (1.77)  Acc@1: 53.1250 (53.1250)  Acc@5: 85.9375 (85.9375)
Valid: 22 [  50/390]  Loss: 1.398 (1.78)  Acc@1: 59.3750 (54.3505)  Acc@5: 89.0625 (84.1605)
Valid: 22 [ 100/390]  Loss: 1.843 (1.80)  Acc@1: 51.5625 (54.5328)  Acc@5: 81.2500 (83.7407)
Valid: 22 [ 150/390]  Loss: 1.774 (1.80)  Acc@1: 53.1250 (54.5737)  Acc@5: 81.2500 (83.5679)
Valid: 22 [ 200/390]  Loss: 2.471 (1.80)  Acc@1: 45.3125 (54.4698)  Acc@5: 76.5625 (83.4188)
Valid: 22 [ 250/390]  Loss: 2.057 (1.79)  Acc@1: 51.5625 (54.5817)  Acc@5: 82.8125 (83.6716)
Valid: 22 [ 300/390]  Loss: 1.417 (1.78)  Acc@1: 56.2500 (54.5837)  Acc@5: 87.5000 (83.7417)
Valid: 22 [ 350/390]  Loss: 1.723 (1.79)  Acc@1: 54.6875 (54.5807)  Acc@5: 84.3750 (83.5960)
Valid: 22 [ 390/390]  Loss: 2.215 (1.79)  Acc@1: 47.5000 (54.5400)  Acc@5: 77.5000 (83.5040)
valid_acc 54.540000
epoch = 22   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1357, 0.8643],
        [0.1712, 0.8288],
        [0.2208, 0.7792],
        [0.1920, 0.8080],
        [0.1816, 0.8184],
        [0.2679, 0.7321],
        [0.1638, 0.8362],
        [0.1594, 0.8406],
        [0.1087, 0.8913],
        [0.2619, 0.7381],
        [0.3526, 0.6474],
        [0.2255, 0.7745],
        [0.4950, 0.5050],
        [0.5288, 0.4712]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5163, 0.4837],
        [0.2916, 0.7084],
        [0.4804, 0.5196],
        [0.5486, 0.4514],
        [0.1338, 0.8662],
        [0.4987, 0.5013],
        [0.2642, 0.7358],
        [0.2011, 0.7989],
        [0.1429, 0.8571],
        [0.4519, 0.5481],
        [0.5404, 0.4596],
        [0.1258, 0.8742],
        [0.1373, 0.8627],
        [0.2333, 0.7667]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 23 [   0/390]  Loss: 0.9598 (0.960)  Acc@1: 68.7500 (68.7500)  Acc@5: 93.7500 (93.7500)LR: 1.450e-02
Train: 23 [  50/390]  Loss: 1.030 (0.939)  Acc@1: 68.7500 (71.1397)  Acc@5: 93.7500 (94.7917)LR: 1.450e-02
Train: 23 [ 100/390]  Loss: 0.9256 (0.946)  Acc@1: 68.7500 (71.1324)  Acc@5: 93.7500 (94.4307)LR: 1.450e-02
Train: 23 [ 150/390]  Loss: 0.8678 (0.971)  Acc@1: 71.8750 (70.7057)  Acc@5: 96.8750 (93.9156)LR: 1.450e-02
Train: 23 [ 200/390]  Loss: 1.132 (0.982)  Acc@1: 70.3125 (70.2659)  Acc@5: 87.5000 (93.7733)LR: 1.450e-02
Train: 23 [ 250/390]  Loss: 0.8600 (0.983)  Acc@1: 71.8750 (70.1506)  Acc@5: 93.7500 (93.7251)LR: 1.450e-02
Train: 23 [ 300/390]  Loss: 0.8663 (0.998)  Acc@1: 73.4375 (69.8557)  Acc@5: 96.8750 (93.5839)LR: 1.450e-02
Train: 23 [ 350/390]  Loss: 0.9449 (1.01)  Acc@1: 71.8750 (69.5913)  Acc@5: 93.7500 (93.3850)LR: 1.450e-02
Train: 23 [ 390/390]  Loss: 1.079 (1.02)  Acc@1: 65.0000 (69.2000)  Acc@5: 87.5000 (93.2960)LR: 1.450e-02
train_acc 69.200000
Valid: 23 [   0/390]  Loss: 1.721 (1.72)  Acc@1: 62.5000 (62.5000)  Acc@5: 85.9375 (85.9375)
Valid: 23 [  50/390]  Loss: 1.999 (1.75)  Acc@1: 56.2500 (55.8824)  Acc@5: 75.0000 (83.4252)
Valid: 23 [ 100/390]  Loss: 1.871 (1.75)  Acc@1: 53.1250 (55.6621)  Acc@5: 81.2500 (83.7098)
Valid: 23 [ 150/390]  Loss: 1.555 (1.77)  Acc@1: 65.6250 (55.4532)  Acc@5: 84.3750 (83.2781)
Valid: 23 [ 200/390]  Loss: 1.661 (1.76)  Acc@1: 54.6875 (55.2861)  Acc@5: 82.8125 (83.3800)
Valid: 23 [ 250/390]  Loss: 1.771 (1.75)  Acc@1: 56.2500 (55.3536)  Acc@5: 89.0625 (83.5969)
Valid: 23 [ 300/390]  Loss: 1.857 (1.75)  Acc@1: 48.4375 (55.4350)  Acc@5: 85.9375 (83.5704)
Valid: 23 [ 350/390]  Loss: 1.438 (1.74)  Acc@1: 59.3750 (55.4398)  Acc@5: 82.8125 (83.5426)
Valid: 23 [ 390/390]  Loss: 2.377 (1.74)  Acc@1: 45.0000 (55.4880)  Acc@5: 72.5000 (83.6080)
valid_acc 55.488000
epoch = 23   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1268, 0.8732],
        [0.1668, 0.8332],
        [0.2189, 0.7811],
        [0.1863, 0.8137],
        [0.1771, 0.8229],
        [0.2532, 0.7468],
        [0.1577, 0.8423],
        [0.1533, 0.8467],
        [0.0997, 0.9003],
        [0.2525, 0.7475],
        [0.3385, 0.6615],
        [0.2207, 0.7793],
        [0.4942, 0.5058],
        [0.5266, 0.4734]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5128, 0.4872],
        [0.2848, 0.7152],
        [0.4764, 0.5236],
        [0.5446, 0.4554],
        [0.1258, 0.8742],
        [0.4959, 0.5041],
        [0.2548, 0.7452],
        [0.1914, 0.8086],
        [0.1346, 0.8654],
        [0.4478, 0.5522],
        [0.5375, 0.4625],
        [0.1181, 0.8819],
        [0.1287, 0.8713],
        [0.2278, 0.7722]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 24 [   0/390]  Loss: 0.7453 (0.745)  Acc@1: 79.6875 (79.6875)  Acc@5: 96.8750 (96.8750)LR: 1.375e-02
Train: 24 [  50/390]  Loss: 1.015 (0.876)  Acc@1: 70.3125 (73.7439)  Acc@5: 93.7500 (94.9755)LR: 1.375e-02
Train: 24 [ 100/390]  Loss: 0.6494 (0.911)  Acc@1: 82.8125 (72.8496)  Acc@5: 96.8750 (94.5080)LR: 1.375e-02
Train: 24 [ 150/390]  Loss: 1.010 (0.931)  Acc@1: 68.7500 (72.2372)  Acc@5: 90.6250 (94.3191)LR: 1.375e-02
Train: 24 [ 200/390]  Loss: 1.049 (0.940)  Acc@1: 70.3125 (71.8361)  Acc@5: 90.6250 (94.3252)LR: 1.375e-02
Train: 24 [ 250/390]  Loss: 1.178 (0.948)  Acc@1: 68.7500 (71.8127)  Acc@5: 92.1875 (94.2169)LR: 1.375e-02
Train: 24 [ 300/390]  Loss: 1.261 (0.958)  Acc@1: 59.3750 (71.5895)  Acc@5: 90.6250 (94.0459)LR: 1.375e-02
Train: 24 [ 350/390]  Loss: 0.9172 (0.962)  Acc@1: 71.8750 (71.3319)  Acc@5: 95.3125 (94.0037)LR: 1.375e-02
Train: 24 [ 390/390]  Loss: 1.043 (0.970)  Acc@1: 67.5000 (71.1040)  Acc@5: 95.0000 (93.9120)LR: 1.375e-02
train_acc 71.104000
Valid: 24 [   0/390]  Loss: 1.328 (1.33)  Acc@1: 60.9375 (60.9375)  Acc@5: 87.5000 (87.5000)
Valid: 24 [  50/390]  Loss: 1.881 (1.78)  Acc@1: 59.3750 (54.6569)  Acc@5: 87.5000 (84.6814)
Valid: 24 [ 100/390]  Loss: 1.922 (1.81)  Acc@1: 57.8125 (54.2079)  Acc@5: 78.1250 (84.6535)
Valid: 24 [ 150/390]  Loss: 2.096 (1.83)  Acc@1: 62.5000 (53.9321)  Acc@5: 82.8125 (84.0646)
Valid: 24 [ 200/390]  Loss: 2.082 (1.82)  Acc@1: 50.0000 (54.4854)  Acc@5: 81.2500 (84.1496)
Valid: 24 [ 250/390]  Loss: 1.579 (1.82)  Acc@1: 60.9375 (54.4323)  Acc@5: 87.5000 (84.0762)
Valid: 24 [ 300/390]  Loss: 1.617 (1.82)  Acc@1: 51.5625 (54.7238)  Acc@5: 85.9375 (83.9234)
Valid: 24 [ 350/390]  Loss: 1.973 (1.82)  Acc@1: 53.1250 (54.8166)  Acc@5: 85.9375 (83.9432)
Valid: 24 [ 390/390]  Loss: 2.111 (1.81)  Acc@1: 47.5000 (54.9680)  Acc@5: 80.0000 (83.9440)
valid_acc 54.968000
epoch = 24   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1189, 0.8811],
        [0.1619, 0.8381],
        [0.2168, 0.7832],
        [0.1821, 0.8179],
        [0.1716, 0.8284],
        [0.2400, 0.7600],
        [0.1565, 0.8435],
        [0.1499, 0.8501],
        [0.0938, 0.9062],
        [0.2459, 0.7541],
        [0.3211, 0.6789],
        [0.2202, 0.7798],
        [0.4919, 0.5081],
        [0.5284, 0.4716]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5089, 0.4911],
        [0.2793, 0.7207],
        [0.4729, 0.5271],
        [0.5464, 0.4536],
        [0.1194, 0.8806],
        [0.4914, 0.5086],
        [0.2452, 0.7548],
        [0.1870, 0.8130],
        [0.1303, 0.8697],
        [0.4430, 0.5570],
        [0.5379, 0.4621],
        [0.1134, 0.8866],
        [0.1233, 0.8767],
        [0.2268, 0.7732]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 25 [   0/390]  Loss: 0.8442 (0.844)  Acc@1: 70.3125 (70.3125)  Acc@5: 96.8750 (96.8750)LR: 1.300e-02
Train: 25 [  50/390]  Loss: 0.7061 (0.842)  Acc@1: 76.5625 (73.4988)  Acc@5: 96.8750 (95.7414)LR: 1.300e-02
Train: 25 [ 100/390]  Loss: 0.9475 (0.872)  Acc@1: 73.4375 (73.1126)  Acc@5: 92.1875 (94.8639)LR: 1.300e-02
Train: 25 [ 150/390]  Loss: 0.8558 (0.880)  Acc@1: 71.8750 (73.1064)  Acc@5: 100.0000 (94.8882)LR: 1.300e-02
Train: 25 [ 200/390]  Loss: 1.025 (0.891)  Acc@1: 68.7500 (73.0488)  Acc@5: 92.1875 (94.6751)LR: 1.300e-02
Train: 25 [ 250/390]  Loss: 0.8915 (0.907)  Acc@1: 75.0000 (72.5909)  Acc@5: 93.7500 (94.4161)LR: 1.300e-02
Train: 25 [ 300/390]  Loss: 0.8646 (0.920)  Acc@1: 76.5625 (72.2851)  Acc@5: 95.3125 (94.2795)LR: 1.300e-02
Train: 25 [ 350/390]  Loss: 1.065 (0.926)  Acc@1: 70.3125 (72.0798)  Acc@5: 92.1875 (94.2397)LR: 1.300e-02
Train: 25 [ 390/390]  Loss: 1.282 (0.935)  Acc@1: 65.0000 (71.7760)  Acc@5: 92.5000 (94.1040)LR: 1.300e-02
train_acc 71.776000
Valid: 25 [   0/390]  Loss: 1.920 (1.92)  Acc@1: 53.1250 (53.1250)  Acc@5: 78.1250 (78.1250)
Valid: 25 [  50/390]  Loss: 1.884 (1.82)  Acc@1: 59.3750 (55.6066)  Acc@5: 82.8125 (83.5478)
Valid: 25 [ 100/390]  Loss: 2.008 (1.79)  Acc@1: 54.6875 (55.9715)  Acc@5: 79.6875 (83.8800)
Valid: 25 [ 150/390]  Loss: 1.597 (1.77)  Acc@1: 57.8125 (56.1051)  Acc@5: 85.9375 (83.9507)
Valid: 25 [ 200/390]  Loss: 1.713 (1.79)  Acc@1: 57.8125 (55.8769)  Acc@5: 82.8125 (83.5821)
Valid: 25 [ 250/390]  Loss: 1.770 (1.79)  Acc@1: 59.3750 (55.5964)  Acc@5: 76.5625 (83.6404)
Valid: 25 [ 300/390]  Loss: 1.559 (1.79)  Acc@1: 64.0625 (55.6375)  Acc@5: 90.6250 (83.6794)
Valid: 25 [ 350/390]  Loss: 1.781 (1.80)  Acc@1: 54.6875 (55.5511)  Acc@5: 82.8125 (83.7117)
Valid: 25 [ 390/390]  Loss: 2.068 (1.80)  Acc@1: 55.0000 (55.5200)  Acc@5: 80.0000 (83.7520)
valid_acc 55.520000
epoch = 25   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('max_pool_3x3', 1), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1129, 0.8871],
        [0.1570, 0.8430],
        [0.2141, 0.7859],
        [0.1805, 0.8195],
        [0.1659, 0.8341],
        [0.2275, 0.7725],
        [0.1557, 0.8443],
        [0.1489, 0.8511],
        [0.0900, 0.9100],
        [0.2398, 0.7602],
        [0.3035, 0.6965],
        [0.2202, 0.7798],
        [0.4888, 0.5112],
        [0.5312, 0.4688]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5057, 0.4943],
        [0.2716, 0.7284],
        [0.4673, 0.5327],
        [0.5410, 0.4590],
        [0.1142, 0.8858],
        [0.4870, 0.5130],
        [0.2349, 0.7651],
        [0.1780, 0.8220],
        [0.1238, 0.8762],
        [0.4381, 0.5619],
        [0.5330, 0.4670],
        [0.1091, 0.8909],
        [0.1155, 0.8845],
        [0.2218, 0.7782]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 26 [   0/390]  Loss: 0.7098 (0.710)  Acc@1: 75.0000 (75.0000)  Acc@5: 100.0000 (100.0000)LR: 1.225e-02
Train: 26 [  50/390]  Loss: 0.9819 (0.856)  Acc@1: 68.7500 (74.0196)  Acc@5: 95.3125 (95.2819)LR: 1.225e-02
Train: 26 [ 100/390]  Loss: 0.9846 (0.870)  Acc@1: 68.7500 (73.3137)  Acc@5: 93.7500 (95.3589)LR: 1.225e-02
Train: 26 [ 150/390]  Loss: 1.002 (0.868)  Acc@1: 71.8750 (73.4272)  Acc@5: 92.1875 (95.3022)LR: 1.225e-02
Train: 26 [ 200/390]  Loss: 0.8752 (0.863)  Acc@1: 71.8750 (73.6241)  Acc@5: 92.1875 (95.1959)LR: 1.225e-02
Train: 26 [ 250/390]  Loss: 1.051 (0.870)  Acc@1: 67.1875 (73.3441)  Acc@5: 92.1875 (95.1506)LR: 1.225e-02
Train: 26 [ 300/390]  Loss: 0.9554 (0.880)  Acc@1: 71.8750 (73.0066)  Acc@5: 90.6250 (95.0166)LR: 1.225e-02
Train: 26 [ 350/390]  Loss: 0.6739 (0.885)  Acc@1: 82.8125 (73.0057)  Acc@5: 95.3125 (94.9030)LR: 1.225e-02
Train: 26 [ 390/390]  Loss: 0.9046 (0.896)  Acc@1: 70.0000 (72.6680)  Acc@5: 97.5000 (94.7760)LR: 1.225e-02
train_acc 72.668000
Valid: 26 [   0/390]  Loss: 1.544 (1.54)  Acc@1: 59.3750 (59.3750)  Acc@5: 89.0625 (89.0625)
Valid: 26 [  50/390]  Loss: 1.651 (1.82)  Acc@1: 59.3750 (55.5760)  Acc@5: 87.5000 (83.3027)
Valid: 26 [ 100/390]  Loss: 1.783 (1.84)  Acc@1: 53.1250 (54.9041)  Acc@5: 84.3750 (83.0136)
Valid: 26 [ 150/390]  Loss: 1.644 (1.83)  Acc@1: 62.5000 (54.8634)  Acc@5: 85.9375 (83.2368)
Valid: 26 [ 200/390]  Loss: 1.591 (1.84)  Acc@1: 54.6875 (54.9363)  Acc@5: 85.9375 (83.2323)
Valid: 26 [ 250/390]  Loss: 1.555 (1.82)  Acc@1: 59.3750 (55.1980)  Acc@5: 87.5000 (83.4350)
Valid: 26 [ 300/390]  Loss: 1.572 (1.81)  Acc@1: 54.6875 (55.3987)  Acc@5: 89.0625 (83.5444)
Valid: 26 [ 350/390]  Loss: 1.839 (1.81)  Acc@1: 51.5625 (55.5110)  Acc@5: 82.8125 (83.5559)
Valid: 26 [ 390/390]  Loss: 1.883 (1.81)  Acc@1: 55.0000 (55.4000)  Acc@5: 87.5000 (83.6080)
valid_acc 55.400000
epoch = 26   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.1053, 0.8947],
        [0.1531, 0.8469],
        [0.2122, 0.7878],
        [0.1756, 0.8244],
        [0.1594, 0.8406],
        [0.2185, 0.7815],
        [0.1540, 0.8460],
        [0.1441, 0.8559],
        [0.0847, 0.9153],
        [0.2313, 0.7687],
        [0.2896, 0.7104],
        [0.2210, 0.7790],
        [0.4865, 0.5135],
        [0.5388, 0.4612]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5020, 0.4980],
        [0.2611, 0.7389],
        [0.4603, 0.5397],
        [0.5390, 0.4610],
        [0.1098, 0.8902],
        [0.4805, 0.5195],
        [0.2233, 0.7767],
        [0.1709, 0.8291],
        [0.1177, 0.8823],
        [0.4313, 0.5687],
        [0.5265, 0.4735],
        [0.1029, 0.8971],
        [0.1076, 0.8924],
        [0.2186, 0.7814]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 27 [   0/390]  Loss: 0.5078 (0.508)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 1.150e-02
Train: 27 [  50/390]  Loss: 0.8467 (0.777)  Acc@1: 75.0000 (76.2868)  Acc@5: 100.0000 (96.3848)LR: 1.150e-02
Train: 27 [ 100/390]  Loss: 1.063 (0.788)  Acc@1: 67.1875 (76.2531)  Acc@5: 95.3125 (96.0705)LR: 1.150e-02
Train: 27 [ 150/390]  Loss: 0.7878 (0.801)  Acc@1: 73.4375 (75.6829)  Acc@5: 98.4375 (95.8195)LR: 1.150e-02
Train: 27 [ 200/390]  Loss: 0.9356 (0.819)  Acc@1: 67.1875 (75.1866)  Acc@5: 93.7500 (95.6157)LR: 1.150e-02
Train: 27 [ 250/390]  Loss: 0.8007 (0.824)  Acc@1: 76.5625 (74.9875)  Acc@5: 96.8750 (95.5304)LR: 1.150e-02
Train: 27 [ 300/390]  Loss: 0.8942 (0.828)  Acc@1: 68.7500 (74.8079)  Acc@5: 93.7500 (95.4630)LR: 1.150e-02
Train: 27 [ 350/390]  Loss: 0.9729 (0.834)  Acc@1: 67.1875 (74.5281)  Acc@5: 96.8750 (95.4639)LR: 1.150e-02
Train: 27 [ 390/390]  Loss: 1.155 (0.837)  Acc@1: 62.5000 (74.3440)  Acc@5: 90.0000 (95.4120)LR: 1.150e-02
train_acc 74.344000
Valid: 27 [   0/390]  Loss: 1.598 (1.60)  Acc@1: 56.2500 (56.2500)  Acc@5: 89.0625 (89.0625)
Valid: 27 [  50/390]  Loss: 1.857 (1.74)  Acc@1: 48.4375 (55.8211)  Acc@5: 84.3750 (84.7733)
Valid: 27 [ 100/390]  Loss: 2.086 (1.76)  Acc@1: 53.1250 (56.0334)  Acc@5: 84.3750 (84.7463)
Valid: 27 [ 150/390]  Loss: 1.983 (1.77)  Acc@1: 57.8125 (56.1569)  Acc@5: 79.6875 (84.9027)
Valid: 27 [ 200/390]  Loss: 1.497 (1.78)  Acc@1: 57.8125 (56.1489)  Acc@5: 90.6250 (84.9269)
Valid: 27 [ 250/390]  Loss: 1.248 (1.76)  Acc@1: 62.5000 (56.6609)  Acc@5: 90.6250 (84.9913)
Valid: 27 [ 300/390]  Loss: 1.680 (1.75)  Acc@1: 62.5000 (56.8989)  Acc@5: 85.9375 (85.0395)
Valid: 27 [ 350/390]  Loss: 2.012 (1.76)  Acc@1: 54.6875 (56.8599)  Acc@5: 81.2500 (84.8869)
Valid: 27 [ 390/390]  Loss: 1.319 (1.76)  Acc@1: 72.5000 (56.8720)  Acc@5: 80.0000 (84.7720)
valid_acc 56.872000
epoch = 27   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0993, 0.9007],
        [0.1498, 0.8502],
        [0.2130, 0.7870],
        [0.1705, 0.8295],
        [0.1550, 0.8450],
        [0.2087, 0.7913],
        [0.1521, 0.8479],
        [0.1422, 0.8578],
        [0.0793, 0.9207],
        [0.2296, 0.7704],
        [0.2757, 0.7243],
        [0.2219, 0.7781],
        [0.4875, 0.5125],
        [0.5406, 0.4594]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5003, 0.4997],
        [0.2498, 0.7502],
        [0.4560, 0.5440],
        [0.5359, 0.4641],
        [0.1062, 0.8938],
        [0.4784, 0.5216],
        [0.2158, 0.7842],
        [0.1679, 0.8321],
        [0.1118, 0.8882],
        [0.4258, 0.5742],
        [0.5202, 0.4798],
        [0.0986, 0.9014],
        [0.1038, 0.8962],
        [0.2150, 0.7850]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 28 [   0/390]  Loss: 0.7012 (0.701)  Acc@1: 78.1250 (78.1250)  Acc@5: 98.4375 (98.4375)LR: 1.075e-02
Train: 28 [  50/390]  Loss: 0.7064 (0.743)  Acc@1: 81.2500 (77.8186)  Acc@5: 93.7500 (96.0172)LR: 1.075e-02
Train: 28 [ 100/390]  Loss: 0.7367 (0.747)  Acc@1: 75.0000 (76.9183)  Acc@5: 98.4375 (96.2717)LR: 1.075e-02
Train: 28 [ 150/390]  Loss: 0.7994 (0.770)  Acc@1: 79.6875 (76.3866)  Acc@5: 95.3125 (96.0679)LR: 1.075e-02
Train: 28 [ 200/390]  Loss: 0.9243 (0.777)  Acc@1: 82.8125 (76.2904)  Acc@5: 96.8750 (95.9966)LR: 1.075e-02
Train: 28 [ 250/390]  Loss: 0.7918 (0.786)  Acc@1: 71.8750 (75.8840)  Acc@5: 98.4375 (95.9475)LR: 1.075e-02
Train: 28 [ 300/390]  Loss: 0.7721 (0.790)  Acc@1: 79.6875 (75.8202)  Acc@5: 98.4375 (95.8472)LR: 1.075e-02
Train: 28 [ 350/390]  Loss: 0.8141 (0.796)  Acc@1: 70.3125 (75.6499)  Acc@5: 98.4375 (95.8111)LR: 1.075e-02
Train: 28 [ 390/390]  Loss: 0.8601 (0.801)  Acc@1: 72.5000 (75.5360)  Acc@5: 90.0000 (95.7480)LR: 1.075e-02
train_acc 75.536000
Valid: 28 [   0/390]  Loss: 1.695 (1.70)  Acc@1: 57.8125 (57.8125)  Acc@5: 87.5000 (87.5000)
Valid: 28 [  50/390]  Loss: 1.578 (1.85)  Acc@1: 60.9375 (55.7598)  Acc@5: 89.0625 (83.7316)
Valid: 28 [ 100/390]  Loss: 1.928 (1.80)  Acc@1: 57.8125 (56.7141)  Acc@5: 76.5625 (84.2667)
Valid: 28 [ 150/390]  Loss: 2.088 (1.80)  Acc@1: 50.0000 (57.0985)  Acc@5: 85.9375 (84.2094)
Valid: 28 [ 200/390]  Loss: 1.923 (1.83)  Acc@1: 54.6875 (56.5143)  Acc@5: 85.9375 (84.1729)
Valid: 28 [ 250/390]  Loss: 2.379 (1.82)  Acc@1: 53.1250 (56.6360)  Acc@5: 76.5625 (84.1696)
Valid: 28 [ 300/390]  Loss: 2.113 (1.82)  Acc@1: 54.6875 (56.5096)  Acc@5: 76.5625 (83.9182)
Valid: 28 [ 350/390]  Loss: 2.016 (1.82)  Acc@1: 53.1250 (56.5260)  Acc@5: 78.1250 (84.0100)
Valid: 28 [ 390/390]  Loss: 1.835 (1.82)  Acc@1: 62.5000 (56.4760)  Acc@5: 87.5000 (84.0480)
valid_acc 56.476000
epoch = 28   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('dil_conv_3x3', 2), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0935, 0.9065],
        [0.1444, 0.8556],
        [0.2147, 0.7853],
        [0.1665, 0.8335],
        [0.1503, 0.8497],
        [0.1981, 0.8019],
        [0.1486, 0.8514],
        [0.1402, 0.8598],
        [0.0748, 0.9252],
        [0.2268, 0.7732],
        [0.2593, 0.7407],
        [0.2201, 0.7799],
        [0.4882, 0.5118],
        [0.5430, 0.4570]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4998, 0.5002],
        [0.2384, 0.7616],
        [0.4527, 0.5473],
        [0.5328, 0.4672],
        [0.1026, 0.8974],
        [0.4785, 0.5215],
        [0.2043, 0.7957],
        [0.1641, 0.8359],
        [0.1061, 0.8939],
        [0.4198, 0.5802],
        [0.5161, 0.4839],
        [0.0946, 0.9054],
        [0.0973, 0.9027],
        [0.2109, 0.7891]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 29 [   0/390]  Loss: 0.5968 (0.597)  Acc@1: 76.5625 (76.5625)  Acc@5: 98.4375 (98.4375)LR: 1.002e-02
Train: 29 [  50/390]  Loss: 0.5508 (0.727)  Acc@1: 84.3750 (77.5735)  Acc@5: 98.4375 (96.3848)LR: 1.002e-02
Train: 29 [ 100/390]  Loss: 0.7550 (0.729)  Acc@1: 76.5625 (77.6300)  Acc@5: 96.8750 (96.3954)LR: 1.002e-02
Train: 29 [ 150/390]  Loss: 0.6731 (0.727)  Acc@1: 81.2500 (77.4214)  Acc@5: 100.0000 (96.5542)LR: 1.002e-02
Train: 29 [ 200/390]  Loss: 0.7924 (0.729)  Acc@1: 73.4375 (77.2544)  Acc@5: 95.3125 (96.5096)LR: 1.002e-02
Train: 29 [ 250/390]  Loss: 0.7641 (0.743)  Acc@1: 78.1250 (76.9920)  Acc@5: 96.8750 (96.4206)LR: 1.002e-02
Train: 29 [ 300/390]  Loss: 0.8410 (0.757)  Acc@1: 73.4375 (76.6559)  Acc@5: 98.4375 (96.3092)LR: 1.002e-02
Train: 29 [ 350/390]  Loss: 0.9669 (0.766)  Acc@1: 71.8750 (76.4156)  Acc@5: 92.1875 (96.1939)LR: 1.002e-02
Train: 29 [ 390/390]  Loss: 0.8884 (0.770)  Acc@1: 80.0000 (76.2960)  Acc@5: 95.0000 (96.1000)LR: 1.002e-02
train_acc 76.296000
Valid: 29 [   0/390]  Loss: 1.680 (1.68)  Acc@1: 62.5000 (62.5000)  Acc@5: 81.2500 (81.2500)
Valid: 29 [  50/390]  Loss: 1.745 (1.81)  Acc@1: 56.2500 (55.9130)  Acc@5: 85.9375 (84.2218)
Valid: 29 [ 100/390]  Loss: 1.874 (1.82)  Acc@1: 53.1250 (56.7450)  Acc@5: 85.9375 (83.9418)
Valid: 29 [ 150/390]  Loss: 1.879 (1.80)  Acc@1: 51.5625 (56.8812)  Acc@5: 85.9375 (84.1784)
Valid: 29 [ 200/390]  Loss: 2.012 (1.80)  Acc@1: 48.4375 (56.9729)  Acc@5: 81.2500 (84.2739)
Valid: 29 [ 250/390]  Loss: 2.051 (1.79)  Acc@1: 53.1250 (56.9783)  Acc@5: 85.9375 (84.3065)
Valid: 29 [ 300/390]  Loss: 1.906 (1.80)  Acc@1: 53.1250 (56.8677)  Acc@5: 81.2500 (84.1051)
Valid: 29 [ 350/390]  Loss: 2.070 (1.82)  Acc@1: 54.6875 (56.5794)  Acc@5: 78.1250 (84.1569)
Valid: 29 [ 390/390]  Loss: 1.366 (1.81)  Acc@1: 60.0000 (56.5920)  Acc@5: 85.0000 (84.1360)
valid_acc 56.592000
epoch = 29   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('dil_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('max_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0879, 0.9121],
        [0.1399, 0.8601],
        [0.2134, 0.7866],
        [0.1647, 0.8353],
        [0.1459, 0.8541],
        [0.1865, 0.8135],
        [0.1479, 0.8521],
        [0.1402, 0.8598],
        [0.0710, 0.9290],
        [0.2218, 0.7782],
        [0.2427, 0.7573],
        [0.2235, 0.7765],
        [0.4912, 0.5088],
        [0.5421, 0.4579]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.5012, 0.4988],
        [0.2320, 0.7680],
        [0.4514, 0.5486],
        [0.5323, 0.4677],
        [0.0981, 0.9019],
        [0.4769, 0.5231],
        [0.1934, 0.8066],
        [0.1592, 0.8408],
        [0.1014, 0.8986],
        [0.4168, 0.5832],
        [0.5167, 0.4833],
        [0.0906, 0.9094],
        [0.0923, 0.9077],
        [0.2063, 0.7937]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 30 [   0/390]  Loss: 0.6089 (0.609)  Acc@1: 79.6875 (79.6875)  Acc@5: 100.0000 (100.0000)LR: 9.292e-03
Train: 30 [  50/390]  Loss: 0.7050 (0.650)  Acc@1: 79.6875 (79.8407)  Acc@5: 98.4375 (97.4571)LR: 9.292e-03
Train: 30 [ 100/390]  Loss: 0.8518 (0.661)  Acc@1: 70.3125 (79.0068)  Acc@5: 95.3125 (97.2927)LR: 9.292e-03
Train: 30 [ 150/390]  Loss: 0.6179 (0.686)  Acc@1: 81.2500 (78.6010)  Acc@5: 100.0000 (96.9681)LR: 9.292e-03
Train: 30 [ 200/390]  Loss: 0.7557 (0.694)  Acc@1: 78.1250 (78.4826)  Acc@5: 95.3125 (96.8750)LR: 9.292e-03
Train: 30 [ 250/390]  Loss: 0.6336 (0.698)  Acc@1: 79.6875 (78.2744)  Acc@5: 96.8750 (96.7692)LR: 9.292e-03
Train: 30 [ 300/390]  Loss: 0.8490 (0.709)  Acc@1: 71.8750 (77.9018)  Acc@5: 95.3125 (96.6258)LR: 9.292e-03
Train: 30 [ 350/390]  Loss: 0.6019 (0.717)  Acc@1: 82.8125 (77.8223)  Acc@5: 96.8750 (96.5189)LR: 9.292e-03
Train: 30 [ 390/390]  Loss: 1.478 (0.722)  Acc@1: 55.0000 (77.6280)  Acc@5: 85.0000 (96.4480)LR: 9.292e-03
train_acc 77.628000
Valid: 30 [   0/390]  Loss: 1.417 (1.42)  Acc@1: 59.3750 (59.3750)  Acc@5: 89.0625 (89.0625)
Valid: 30 [  50/390]  Loss: 1.896 (1.79)  Acc@1: 60.9375 (56.4951)  Acc@5: 84.3750 (84.6814)
Valid: 30 [ 100/390]  Loss: 2.029 (1.75)  Acc@1: 57.8125 (56.7450)  Acc@5: 82.8125 (84.8082)
Valid: 30 [ 150/390]  Loss: 1.440 (1.74)  Acc@1: 65.6250 (57.2020)  Acc@5: 89.0625 (84.9027)
Valid: 30 [ 200/390]  Loss: 1.462 (1.76)  Acc@1: 68.7500 (57.0351)  Acc@5: 84.3750 (84.7326)
Valid: 30 [ 250/390]  Loss: 1.887 (1.75)  Acc@1: 54.6875 (57.2336)  Acc@5: 81.2500 (84.7298)
Valid: 30 [ 300/390]  Loss: 2.369 (1.76)  Acc@1: 45.3125 (56.9456)  Acc@5: 70.3125 (84.6190)
Valid: 30 [ 350/390]  Loss: 1.851 (1.77)  Acc@1: 56.2500 (56.9266)  Acc@5: 79.6875 (84.7000)
Valid: 30 [ 390/390]  Loss: 1.574 (1.76)  Acc@1: 57.5000 (56.8720)  Acc@5: 87.5000 (84.6520)
valid_acc 56.872000
epoch = 30   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('dil_conv_3x3', 2)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0826, 0.9174],
        [0.1385, 0.8615],
        [0.2121, 0.7879],
        [0.1612, 0.8388],
        [0.1421, 0.8579],
        [0.1760, 0.8240],
        [0.1454, 0.8546],
        [0.1391, 0.8609],
        [0.0668, 0.9332],
        [0.2187, 0.7813],
        [0.2291, 0.7709],
        [0.2269, 0.7731],
        [0.4931, 0.5069],
        [0.5394, 0.4606]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4984, 0.5016],
        [0.2283, 0.7717],
        [0.4484, 0.5516],
        [0.5286, 0.4714],
        [0.0937, 0.9063],
        [0.4704, 0.5296],
        [0.1841, 0.8159],
        [0.1545, 0.8455],
        [0.0971, 0.9029],
        [0.4107, 0.5893],
        [0.5131, 0.4869],
        [0.0860, 0.9140],
        [0.0878, 0.9122],
        [0.2029, 0.7971]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 31 [   0/390]  Loss: 0.5343 (0.534)  Acc@1: 78.1250 (78.1250)  Acc@5: 98.4375 (98.4375)LR: 8.583e-03
Train: 31 [  50/390]  Loss: 0.5571 (0.615)  Acc@1: 84.3750 (80.6679)  Acc@5: 98.4375 (97.9779)LR: 8.583e-03
Train: 31 [ 100/390]  Loss: 0.4880 (0.622)  Acc@1: 87.5000 (80.4920)  Acc@5: 100.0000 (97.7259)LR: 8.583e-03
Train: 31 [ 150/390]  Loss: 0.5690 (0.635)  Acc@1: 84.3750 (80.3601)  Acc@5: 100.0000 (97.4752)LR: 8.583e-03
Train: 31 [ 200/390]  Loss: 0.8212 (0.642)  Acc@1: 76.5625 (80.2394)  Acc@5: 92.1875 (97.3647)LR: 8.583e-03
Train: 31 [ 250/390]  Loss: 0.7423 (0.651)  Acc@1: 75.0000 (79.7871)  Acc@5: 96.8750 (97.2734)LR: 8.583e-03
Train: 31 [ 300/390]  Loss: 1.008 (0.657)  Acc@1: 68.7500 (79.6096)  Acc@5: 92.1875 (97.2072)LR: 8.583e-03
Train: 31 [ 350/390]  Loss: 0.6711 (0.669)  Acc@1: 76.5625 (79.2468)  Acc@5: 98.4375 (97.0931)LR: 8.583e-03
Train: 31 [ 390/390]  Loss: 0.7852 (0.679)  Acc@1: 85.0000 (78.9440)  Acc@5: 95.0000 (97.0400)LR: 8.583e-03
train_acc 78.944000
Valid: 31 [   0/390]  Loss: 1.925 (1.92)  Acc@1: 50.0000 (50.0000)  Acc@5: 85.9375 (85.9375)
Valid: 31 [  50/390]  Loss: 2.043 (1.77)  Acc@1: 57.8125 (57.9044)  Acc@5: 79.6875 (84.9877)
Valid: 31 [ 100/390]  Loss: 2.445 (1.79)  Acc@1: 51.5625 (57.3639)  Acc@5: 76.5625 (85.2259)
Valid: 31 [ 150/390]  Loss: 1.824 (1.80)  Acc@1: 51.5625 (57.3779)  Acc@5: 82.8125 (85.0683)
Valid: 31 [ 200/390]  Loss: 1.224 (1.80)  Acc@1: 67.1875 (57.4471)  Acc@5: 92.1875 (84.8725)
Valid: 31 [ 250/390]  Loss: 1.195 (1.79)  Acc@1: 68.7500 (57.4701)  Acc@5: 93.7500 (84.9228)
Valid: 31 [ 300/390]  Loss: 2.070 (1.80)  Acc@1: 56.2500 (57.4595)  Acc@5: 78.1250 (84.7436)
Valid: 31 [ 350/390]  Loss: 1.575 (1.79)  Acc@1: 60.9375 (57.5855)  Acc@5: 90.6250 (84.8780)
Valid: 31 [ 390/390]  Loss: 1.441 (1.79)  Acc@1: 55.0000 (57.5400)  Acc@5: 95.0000 (84.8080)
valid_acc 57.540000
epoch = 31   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('sep_conv_3x3', 0), ('skip_connect', 1)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0780, 0.9220],
        [0.1360, 0.8640],
        [0.2128, 0.7872],
        [0.1599, 0.8401],
        [0.1405, 0.8595],
        [0.1647, 0.8353],
        [0.1440, 0.8560],
        [0.1374, 0.8626],
        [0.0638, 0.9362],
        [0.2116, 0.7884],
        [0.2137, 0.7863],
        [0.2306, 0.7694],
        [0.4895, 0.5105],
        [0.5337, 0.4663]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4968, 0.5032],
        [0.2192, 0.7808],
        [0.4458, 0.5542],
        [0.5243, 0.4757],
        [0.0885, 0.9115],
        [0.4676, 0.5324],
        [0.1727, 0.8273],
        [0.1494, 0.8506],
        [0.0926, 0.9074],
        [0.4048, 0.5952],
        [0.5060, 0.4940],
        [0.0830, 0.9170],
        [0.0833, 0.9167],
        [0.1997, 0.8003]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 32 [   0/390]  Loss: 0.5016 (0.502)  Acc@1: 85.9375 (85.9375)  Acc@5: 96.8750 (96.8750)LR: 7.891e-03
Train: 32 [  50/390]  Loss: 0.4808 (0.619)  Acc@1: 81.2500 (80.7292)  Acc@5: 96.8750 (97.2733)LR: 7.891e-03
Train: 32 [ 100/390]  Loss: 0.6625 (0.618)  Acc@1: 79.6875 (80.7704)  Acc@5: 96.8750 (97.3855)LR: 7.891e-03
Train: 32 [ 150/390]  Loss: 0.5915 (0.619)  Acc@1: 78.1250 (80.6498)  Acc@5: 96.8750 (97.5476)LR: 7.891e-03
Train: 32 [ 200/390]  Loss: 0.5021 (0.613)  Acc@1: 84.3750 (80.9468)  Acc@5: 98.4375 (97.6057)LR: 7.891e-03
Train: 32 [ 250/390]  Loss: 0.6144 (0.619)  Acc@1: 84.3750 (80.7022)  Acc@5: 98.4375 (97.6220)LR: 7.891e-03
Train: 32 [ 300/390]  Loss: 0.7195 (0.626)  Acc@1: 78.1250 (80.4662)  Acc@5: 96.8750 (97.4772)LR: 7.891e-03
Train: 32 [ 350/390]  Loss: 0.5340 (0.634)  Acc@1: 85.9375 (80.1816)  Acc@5: 100.0000 (97.3691)LR: 7.891e-03
Train: 32 [ 390/390]  Loss: 0.6739 (0.638)  Acc@1: 82.5000 (80.1160)  Acc@5: 95.0000 (97.3040)LR: 7.891e-03
train_acc 80.116000
Valid: 32 [   0/390]  Loss: 1.357 (1.36)  Acc@1: 68.7500 (68.7500)  Acc@5: 89.0625 (89.0625)
Valid: 32 [  50/390]  Loss: 2.035 (1.87)  Acc@1: 48.4375 (57.4755)  Acc@5: 84.3750 (84.1605)
Valid: 32 [ 100/390]  Loss: 1.519 (1.85)  Acc@1: 50.0000 (57.7351)  Acc@5: 90.6250 (84.1584)
Valid: 32 [ 150/390]  Loss: 2.244 (1.82)  Acc@1: 56.2500 (58.0195)  Acc@5: 79.6875 (84.7165)
Valid: 32 [ 200/390]  Loss: 1.789 (1.81)  Acc@1: 62.5000 (57.9680)  Acc@5: 81.2500 (84.8570)
Valid: 32 [ 250/390]  Loss: 1.935 (1.81)  Acc@1: 59.3750 (57.7938)  Acc@5: 81.2500 (84.8170)
Valid: 32 [ 300/390]  Loss: 1.856 (1.81)  Acc@1: 53.1250 (57.7087)  Acc@5: 82.8125 (84.7799)
Valid: 32 [ 350/390]  Loss: 1.038 (1.80)  Acc@1: 70.3125 (57.7724)  Acc@5: 93.7500 (84.8958)
Valid: 32 [ 390/390]  Loss: 1.745 (1.79)  Acc@1: 55.0000 (57.9360)  Acc@5: 82.5000 (85.0400)
valid_acc 57.936000
epoch = 32   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 2), ('dil_conv_5x5', 3)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0738, 0.9262],
        [0.1338, 0.8662],
        [0.2112, 0.7888],
        [0.1575, 0.8425],
        [0.1379, 0.8621],
        [0.1542, 0.8458],
        [0.1422, 0.8578],
        [0.1395, 0.8605],
        [0.0607, 0.9393],
        [0.2087, 0.7913],
        [0.2019, 0.7981],
        [0.2346, 0.7654],
        [0.4918, 0.5082],
        [0.5323, 0.4677]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4933, 0.5067],
        [0.2106, 0.7894],
        [0.4398, 0.5602],
        [0.5181, 0.4819],
        [0.0846, 0.9154],
        [0.4643, 0.5357],
        [0.1639, 0.8361],
        [0.1440, 0.8560],
        [0.0890, 0.9110],
        [0.3988, 0.6012],
        [0.5022, 0.4978],
        [0.0788, 0.9212],
        [0.0793, 0.9207],
        [0.1945, 0.8055]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 33 [   0/390]  Loss: 0.4409 (0.441)  Acc@1: 90.6250 (90.6250)  Acc@5: 96.8750 (96.8750)LR: 7.219e-03
Train: 33 [  50/390]  Loss: 0.3084 (0.530)  Acc@1: 92.1875 (83.4252)  Acc@5: 100.0000 (97.8860)LR: 7.219e-03
Train: 33 [ 100/390]  Loss: 0.4035 (0.553)  Acc@1: 87.5000 (82.7661)  Acc@5: 98.4375 (97.8651)LR: 7.219e-03
Train: 33 [ 150/390]  Loss: 0.4360 (0.552)  Acc@1: 85.9375 (83.1333)  Acc@5: 100.0000 (97.9615)LR: 7.219e-03
Train: 33 [ 200/390]  Loss: 0.4292 (0.560)  Acc@1: 90.6250 (82.9757)  Acc@5: 98.4375 (97.8078)LR: 7.219e-03
Train: 33 [ 250/390]  Loss: 0.4137 (0.571)  Acc@1: 85.9375 (82.5261)  Acc@5: 100.0000 (97.7341)LR: 7.219e-03
Train: 33 [ 300/390]  Loss: 0.7458 (0.575)  Acc@1: 79.6875 (82.3453)  Acc@5: 96.8750 (97.6744)LR: 7.219e-03
Train: 33 [ 350/390]  Loss: 0.9221 (0.577)  Acc@1: 73.4375 (82.1492)  Acc@5: 95.3125 (97.7208)LR: 7.219e-03
Train: 33 [ 390/390]  Loss: 0.4547 (0.580)  Acc@1: 85.0000 (81.9840)  Acc@5: 100.0000 (97.6840)LR: 7.219e-03
train_acc 81.984000
Valid: 33 [   0/390]  Loss: 1.621 (1.62)  Acc@1: 56.2500 (56.2500)  Acc@5: 89.0625 (89.0625)
Valid: 33 [  50/390]  Loss: 1.265 (1.82)  Acc@1: 65.6250 (57.7819)  Acc@5: 90.6250 (85.2022)
Valid: 33 [ 100/390]  Loss: 2.768 (1.81)  Acc@1: 45.3125 (57.9827)  Acc@5: 75.0000 (85.3496)
Valid: 33 [ 150/390]  Loss: 1.714 (1.78)  Acc@1: 60.9375 (58.2781)  Acc@5: 85.9375 (85.3270)
Valid: 33 [ 200/390]  Loss: 1.717 (1.80)  Acc@1: 60.9375 (58.0535)  Acc@5: 79.6875 (85.1524)
Valid: 33 [ 250/390]  Loss: 1.586 (1.80)  Acc@1: 60.9375 (58.2358)  Acc@5: 85.9375 (85.2590)
Valid: 33 [ 300/390]  Loss: 1.602 (1.80)  Acc@1: 62.5000 (58.2797)  Acc@5: 85.9375 (85.2990)
Valid: 33 [ 350/390]  Loss: 1.680 (1.80)  Acc@1: 57.8125 (58.2799)  Acc@5: 85.9375 (85.2342)
Valid: 33 [ 390/390]  Loss: 1.384 (1.80)  Acc@1: 65.0000 (58.2680)  Acc@5: 90.0000 (85.2080)
valid_acc 58.268000
epoch = 33   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('sep_conv_3x3', 2), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0709, 0.9291],
        [0.1316, 0.8684],
        [0.2085, 0.7915],
        [0.1565, 0.8435],
        [0.1351, 0.8649],
        [0.1456, 0.8544],
        [0.1417, 0.8583],
        [0.1377, 0.8623],
        [0.0585, 0.9415],
        [0.2024, 0.7976],
        [0.1872, 0.8128],
        [0.2382, 0.7618],
        [0.4957, 0.5043],
        [0.5260, 0.4740]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4877, 0.5123],
        [0.2054, 0.7946],
        [0.4343, 0.5657],
        [0.5184, 0.4816],
        [0.0821, 0.9179],
        [0.4539, 0.5461],
        [0.1590, 0.8410],
        [0.1399, 0.8601],
        [0.0851, 0.9149],
        [0.3890, 0.6110],
        [0.4958, 0.5042],
        [0.0760, 0.9240],
        [0.0750, 0.9250],
        [0.1900, 0.8100]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 34 [   0/390]  Loss: 0.6183 (0.618)  Acc@1: 79.6875 (79.6875)  Acc@5: 100.0000 (100.0000)LR: 6.570e-03
Train: 34 [  50/390]  Loss: 0.5371 (0.507)  Acc@1: 81.2500 (84.9877)  Acc@5: 98.4375 (98.3150)LR: 6.570e-03
Train: 34 [ 100/390]  Loss: 0.6184 (0.508)  Acc@1: 84.3750 (84.5142)  Acc@5: 98.4375 (98.3292)LR: 6.570e-03
Train: 34 [ 150/390]  Loss: 0.4994 (0.509)  Acc@1: 82.8125 (84.1370)  Acc@5: 98.4375 (98.3237)LR: 6.570e-03
Train: 34 [ 200/390]  Loss: 0.5357 (0.514)  Acc@1: 81.2500 (84.0019)  Acc@5: 100.0000 (98.1654)LR: 6.570e-03
Train: 34 [ 250/390]  Loss: 0.3730 (0.519)  Acc@1: 89.0625 (83.6965)  Acc@5: 100.0000 (98.1760)LR: 6.570e-03
Train: 34 [ 300/390]  Loss: 0.5137 (0.526)  Acc@1: 87.5000 (83.4873)  Acc@5: 98.4375 (98.1831)LR: 6.570e-03
Train: 34 [ 350/390]  Loss: 0.6433 (0.537)  Acc@1: 76.5625 (83.2577)  Acc@5: 98.4375 (98.0814)LR: 6.570e-03
Train: 34 [ 390/390]  Loss: 0.5993 (0.543)  Acc@1: 85.0000 (83.1480)  Acc@5: 97.5000 (98.0120)LR: 6.570e-03
train_acc 83.148000
Valid: 34 [   0/390]  Loss: 1.926 (1.93)  Acc@1: 54.6875 (54.6875)  Acc@5: 79.6875 (79.6875)
Valid: 34 [  50/390]  Loss: 1.355 (1.77)  Acc@1: 57.8125 (58.2414)  Acc@5: 90.6250 (85.0184)
Valid: 34 [ 100/390]  Loss: 1.764 (1.76)  Acc@1: 53.1250 (58.3849)  Acc@5: 85.9375 (85.6900)
Valid: 34 [ 150/390]  Loss: 2.117 (1.76)  Acc@1: 56.2500 (58.3195)  Acc@5: 79.6875 (85.4925)
Valid: 34 [ 200/390]  Loss: 1.527 (1.76)  Acc@1: 57.8125 (58.2556)  Acc@5: 92.1875 (85.6188)
Valid: 34 [ 250/390]  Loss: 2.143 (1.78)  Acc@1: 53.1250 (58.0926)  Acc@5: 75.0000 (85.4706)
Valid: 34 [ 300/390]  Loss: 1.967 (1.79)  Acc@1: 51.5625 (58.0253)  Acc@5: 85.9375 (85.2523)
Valid: 34 [ 350/390]  Loss: 2.133 (1.79)  Acc@1: 57.8125 (58.1019)  Acc@5: 82.8125 (85.2252)
Valid: 34 [ 390/390]  Loss: 1.936 (1.78)  Acc@1: 62.5000 (58.2040)  Acc@5: 80.0000 (85.3760)
valid_acc 58.204000
epoch = 34   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0666, 0.9334],
        [0.1292, 0.8708],
        [0.2076, 0.7924],
        [0.1549, 0.8451],
        [0.1340, 0.8660],
        [0.1381, 0.8619],
        [0.1405, 0.8595],
        [0.1394, 0.8606],
        [0.0560, 0.9440],
        [0.1961, 0.8039],
        [0.1757, 0.8243],
        [0.2418, 0.7582],
        [0.4984, 0.5016],
        [0.5219, 0.4781]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4865, 0.5135],
        [0.2010, 0.7990],
        [0.4297, 0.5703],
        [0.5147, 0.4853],
        [0.0783, 0.9217],
        [0.4516, 0.5484],
        [0.1515, 0.8485],
        [0.1353, 0.8647],
        [0.0813, 0.9187],
        [0.3867, 0.6133],
        [0.4916, 0.5084],
        [0.0719, 0.9281],
        [0.0712, 0.9288],
        [0.1873, 0.8127]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 35 [   0/390]  Loss: 0.4731 (0.473)  Acc@1: 85.9375 (85.9375)  Acc@5: 98.4375 (98.4375)LR: 5.947e-03
Train: 35 [  50/390]  Loss: 0.5041 (0.496)  Acc@1: 85.9375 (84.6814)  Acc@5: 100.0000 (98.4988)LR: 5.947e-03
Train: 35 [ 100/390]  Loss: 0.3028 (0.489)  Acc@1: 93.7500 (84.7308)  Acc@5: 100.0000 (98.5458)LR: 5.947e-03
Train: 35 [ 150/390]  Loss: 0.5414 (0.489)  Acc@1: 84.3750 (84.5716)  Acc@5: 95.3125 (98.4789)LR: 5.947e-03
Train: 35 [ 200/390]  Loss: 0.6074 (0.493)  Acc@1: 79.6875 (84.4761)  Acc@5: 95.3125 (98.3598)LR: 5.947e-03
Train: 35 [ 250/390]  Loss: 0.6093 (0.497)  Acc@1: 82.8125 (84.4124)  Acc@5: 98.4375 (98.3815)LR: 5.947e-03
Train: 35 [ 300/390]  Loss: 0.5057 (0.504)  Acc@1: 78.1250 (84.1725)  Acc@5: 98.4375 (98.2818)LR: 5.947e-03
Train: 35 [ 350/390]  Loss: 0.5889 (0.511)  Acc@1: 87.5000 (83.9922)  Acc@5: 100.0000 (98.2372)LR: 5.947e-03
Train: 35 [ 390/390]  Loss: 0.3228 (0.517)  Acc@1: 90.0000 (83.6760)  Acc@5: 100.0000 (98.1840)LR: 5.947e-03
train_acc 83.676000
Valid: 35 [   0/390]  Loss: 1.418 (1.42)  Acc@1: 54.6875 (54.6875)  Acc@5: 90.6250 (90.6250)
Valid: 35 [  50/390]  Loss: 1.729 (1.75)  Acc@1: 62.5000 (59.4056)  Acc@5: 89.0625 (85.2328)
Valid: 35 [ 100/390]  Loss: 1.703 (1.79)  Acc@1: 59.3750 (58.7871)  Acc@5: 82.8125 (84.8855)
Valid: 35 [ 150/390]  Loss: 1.876 (1.79)  Acc@1: 54.6875 (58.9404)  Acc@5: 82.8125 (85.3373)
Valid: 35 [ 200/390]  Loss: 1.477 (1.78)  Acc@1: 65.6250 (58.9086)  Acc@5: 85.9375 (85.4167)
Valid: 35 [ 250/390]  Loss: 1.937 (1.78)  Acc@1: 57.8125 (58.8272)  Acc@5: 82.8125 (85.4395)
Valid: 35 [ 300/390]  Loss: 1.697 (1.79)  Acc@1: 60.9375 (58.7417)  Acc@5: 84.3750 (85.4807)
Valid: 35 [ 350/390]  Loss: 1.428 (1.78)  Acc@1: 65.6250 (58.9744)  Acc@5: 89.0625 (85.5413)
Valid: 35 [ 390/390]  Loss: 1.723 (1.78)  Acc@1: 52.5000 (58.8600)  Acc@5: 90.0000 (85.5520)
valid_acc 58.860000
epoch = 35   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0630, 0.9370],
        [0.1260, 0.8740],
        [0.2034, 0.7966],
        [0.1520, 0.8480],
        [0.1311, 0.8689],
        [0.1302, 0.8698],
        [0.1391, 0.8609],
        [0.1403, 0.8597],
        [0.0545, 0.9455],
        [0.1928, 0.8072],
        [0.1662, 0.8338],
        [0.2427, 0.7573],
        [0.5036, 0.4964],
        [0.5186, 0.4814]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4868, 0.5132],
        [0.1982, 0.8018],
        [0.4286, 0.5714],
        [0.5135, 0.4865],
        [0.0745, 0.9255],
        [0.4520, 0.5480],
        [0.1461, 0.8539],
        [0.1305, 0.8695],
        [0.0794, 0.9206],
        [0.3853, 0.6147],
        [0.4898, 0.5102],
        [0.0693, 0.9307],
        [0.0683, 0.9317],
        [0.1841, 0.8159]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 36 [   0/390]  Loss: 0.4730 (0.473)  Acc@1: 82.8125 (82.8125)  Acc@5: 98.4375 (98.4375)LR: 5.351e-03
Train: 36 [  50/390]  Loss: 0.3124 (0.439)  Acc@1: 92.1875 (86.5809)  Acc@5: 100.0000 (99.0809)LR: 5.351e-03
Train: 36 [ 100/390]  Loss: 0.4174 (0.446)  Acc@1: 85.9375 (86.1541)  Acc@5: 100.0000 (98.9480)LR: 5.351e-03
Train: 36 [ 150/390]  Loss: 0.4999 (0.446)  Acc@1: 84.3750 (86.3721)  Acc@5: 98.4375 (98.8204)LR: 5.351e-03
Train: 36 [ 200/390]  Loss: 0.3075 (0.448)  Acc@1: 89.0625 (86.4583)  Acc@5: 100.0000 (98.7407)LR: 5.351e-03
Train: 36 [ 250/390]  Loss: 0.4414 (0.454)  Acc@1: 85.9375 (86.1616)  Acc@5: 100.0000 (98.7114)LR: 5.351e-03
Train: 36 [ 300/390]  Loss: 0.4876 (0.462)  Acc@1: 82.8125 (85.7506)  Acc@5: 100.0000 (98.6555)LR: 5.351e-03
Train: 36 [ 350/390]  Loss: 0.6742 (0.467)  Acc@1: 75.0000 (85.5680)  Acc@5: 95.3125 (98.5710)LR: 5.351e-03
Train: 36 [ 390/390]  Loss: 0.5202 (0.471)  Acc@1: 85.0000 (85.4240)  Acc@5: 100.0000 (98.5640)LR: 5.351e-03
train_acc 85.424000
Valid: 36 [   0/390]  Loss: 1.981 (1.98)  Acc@1: 54.6875 (54.6875)  Acc@5: 87.5000 (87.5000)
Valid: 36 [  50/390]  Loss: 2.021 (1.84)  Acc@1: 60.9375 (58.6091)  Acc@5: 87.5000 (85.2022)
Valid: 36 [ 100/390]  Loss: 1.735 (1.82)  Acc@1: 62.5000 (58.4313)  Acc@5: 82.8125 (85.6281)
Valid: 36 [ 150/390]  Loss: 1.730 (1.82)  Acc@1: 65.6250 (58.4230)  Acc@5: 85.9375 (85.6892)
Valid: 36 [ 200/390]  Loss: 1.920 (1.81)  Acc@1: 59.3750 (58.6521)  Acc@5: 85.9375 (85.5566)
Valid: 36 [ 250/390]  Loss: 1.204 (1.80)  Acc@1: 68.7500 (58.9890)  Acc@5: 92.1875 (85.6325)
Valid: 36 [ 300/390]  Loss: 1.868 (1.81)  Acc@1: 54.6875 (58.8455)  Acc@5: 84.3750 (85.5326)
Valid: 36 [ 350/390]  Loss: 1.413 (1.81)  Acc@1: 65.6250 (58.7206)  Acc@5: 85.9375 (85.4701)
Valid: 36 [ 390/390]  Loss: 1.498 (1.81)  Acc@1: 55.0000 (58.7040)  Acc@5: 87.5000 (85.4840)
valid_acc 58.704000
epoch = 36   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0605, 0.9395],
        [0.1235, 0.8765],
        [0.2006, 0.7994],
        [0.1499, 0.8501],
        [0.1290, 0.8710],
        [0.1230, 0.8770],
        [0.1366, 0.8634],
        [0.1395, 0.8605],
        [0.0531, 0.9469],
        [0.1869, 0.8131],
        [0.1557, 0.8443],
        [0.2435, 0.7565],
        [0.5062, 0.4938],
        [0.5200, 0.4800]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4813, 0.5187],
        [0.1932, 0.8068],
        [0.4236, 0.5764],
        [0.5098, 0.4902],
        [0.0722, 0.9278],
        [0.4489, 0.5511],
        [0.1408, 0.8592],
        [0.1263, 0.8737],
        [0.0758, 0.9242],
        [0.3794, 0.6206],
        [0.4846, 0.5154],
        [0.0669, 0.9331],
        [0.0646, 0.9354],
        [0.1819, 0.8181]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 37 [   0/390]  Loss: 0.5194 (0.519)  Acc@1: 85.9375 (85.9375)  Acc@5: 95.3125 (95.3125)LR: 4.785e-03
Train: 37 [  50/390]  Loss: 0.2281 (0.395)  Acc@1: 92.1875 (87.7451)  Acc@5: 100.0000 (99.3260)LR: 4.785e-03
Train: 37 [ 100/390]  Loss: 0.5174 (0.404)  Acc@1: 82.8125 (87.5774)  Acc@5: 98.4375 (99.1801)LR: 4.785e-03
Train: 37 [ 150/390]  Loss: 0.3068 (0.411)  Acc@1: 87.5000 (87.3448)  Acc@5: 98.4375 (98.9549)LR: 4.785e-03
Train: 37 [ 200/390]  Loss: 0.4829 (0.413)  Acc@1: 89.0625 (87.2590)  Acc@5: 96.8750 (98.9506)LR: 4.785e-03
Train: 37 [ 250/390]  Loss: 0.3370 (0.418)  Acc@1: 89.0625 (87.0393)  Acc@5: 100.0000 (98.9231)LR: 4.785e-03
Train: 37 [ 300/390]  Loss: 0.4753 (0.426)  Acc@1: 85.9375 (86.7265)  Acc@5: 98.4375 (98.8891)LR: 4.785e-03
Train: 37 [ 350/390]  Loss: 0.4579 (0.426)  Acc@1: 84.3750 (86.6676)  Acc@5: 98.4375 (98.8827)LR: 4.785e-03
Train: 37 [ 390/390]  Loss: 0.4026 (0.425)  Acc@1: 85.0000 (86.6640)  Acc@5: 100.0000 (98.8960)LR: 4.785e-03
train_acc 86.664000
Valid: 37 [   0/390]  Loss: 1.604 (1.60)  Acc@1: 64.0625 (64.0625)  Acc@5: 82.8125 (82.8125)
Valid: 37 [  50/390]  Loss: 1.374 (1.80)  Acc@1: 64.0625 (58.5784)  Acc@5: 92.1875 (85.6618)
Valid: 37 [ 100/390]  Loss: 1.818 (1.80)  Acc@1: 59.3750 (58.4158)  Acc@5: 84.3750 (85.4425)
Valid: 37 [ 150/390]  Loss: 1.465 (1.80)  Acc@1: 64.0625 (58.6507)  Acc@5: 92.1875 (85.4925)
Valid: 37 [ 200/390]  Loss: 1.734 (1.80)  Acc@1: 68.7500 (58.9552)  Acc@5: 81.2500 (85.4089)
Valid: 37 [ 250/390]  Loss: 2.428 (1.79)  Acc@1: 54.6875 (59.1820)  Acc@5: 78.1250 (85.4457)
Valid: 37 [ 300/390]  Loss: 2.121 (1.79)  Acc@1: 51.5625 (59.0947)  Acc@5: 81.2500 (85.5378)
Valid: 37 [ 350/390]  Loss: 2.279 (1.79)  Acc@1: 48.4375 (59.1257)  Acc@5: 79.6875 (85.7060)
Valid: 37 [ 390/390]  Loss: 1.477 (1.79)  Acc@1: 67.5000 (59.4120)  Acc@5: 92.5000 (85.7240)
valid_acc 59.412000
epoch = 37   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0580, 0.9420],
        [0.1210, 0.8790],
        [0.1971, 0.8029],
        [0.1489, 0.8511],
        [0.1264, 0.8736],
        [0.1175, 0.8825],
        [0.1342, 0.8658],
        [0.1400, 0.8600],
        [0.0515, 0.9485],
        [0.1840, 0.8160],
        [0.1486, 0.8514],
        [0.2462, 0.7538],
        [0.5062, 0.4938],
        [0.5191, 0.4809]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4766, 0.5234],
        [0.1894, 0.8106],
        [0.4197, 0.5803],
        [0.5075, 0.4925],
        [0.0684, 0.9316],
        [0.4458, 0.5542],
        [0.1344, 0.8656],
        [0.1226, 0.8774],
        [0.0735, 0.9265],
        [0.3735, 0.6265],
        [0.4813, 0.5187],
        [0.0640, 0.9360],
        [0.0624, 0.9376],
        [0.1771, 0.8229]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 38 [   0/390]  Loss: 0.2726 (0.273)  Acc@1: 92.1875 (92.1875)  Acc@5: 100.0000 (100.0000)LR: 4.252e-03
Train: 38 [  50/390]  Loss: 0.2163 (0.354)  Acc@1: 92.1875 (88.5723)  Acc@5: 100.0000 (99.3566)LR: 4.252e-03
Train: 38 [ 100/390]  Loss: 0.4697 (0.361)  Acc@1: 81.2500 (88.5829)  Acc@5: 100.0000 (99.2729)LR: 4.252e-03
Train: 38 [ 150/390]  Loss: 0.6050 (0.373)  Acc@1: 84.3750 (88.4520)  Acc@5: 96.8750 (99.1308)LR: 4.252e-03
Train: 38 [ 200/390]  Loss: 0.4480 (0.381)  Acc@1: 87.5000 (88.2385)  Acc@5: 98.4375 (99.0594)LR: 4.252e-03
Train: 38 [ 250/390]  Loss: 0.5086 (0.384)  Acc@1: 84.3750 (88.3030)  Acc@5: 96.8750 (99.0289)LR: 4.252e-03
Train: 38 [ 300/390]  Loss: 0.2643 (0.392)  Acc@1: 92.1875 (88.0087)  Acc@5: 100.0000 (98.9877)LR: 4.252e-03
Train: 38 [ 350/390]  Loss: 0.2984 (0.399)  Acc@1: 93.7500 (87.7938)  Acc@5: 100.0000 (98.9672)LR: 4.252e-03
Train: 38 [ 390/390]  Loss: 0.4091 (0.402)  Acc@1: 87.5000 (87.6960)  Acc@5: 100.0000 (98.9680)LR: 4.252e-03
train_acc 87.696000
Valid: 38 [   0/390]  Loss: 1.785 (1.79)  Acc@1: 60.9375 (60.9375)  Acc@5: 89.0625 (89.0625)
Valid: 38 [  50/390]  Loss: 1.784 (1.80)  Acc@1: 65.6250 (60.1716)  Acc@5: 82.8125 (86.3664)
Valid: 38 [ 100/390]  Loss: 2.238 (1.79)  Acc@1: 54.6875 (59.3286)  Acc@5: 81.2500 (86.1696)
Valid: 38 [ 150/390]  Loss: 2.186 (1.78)  Acc@1: 51.5625 (59.2612)  Acc@5: 84.3750 (85.9685)
Valid: 38 [ 200/390]  Loss: 1.563 (1.77)  Acc@1: 60.9375 (59.4216)  Acc@5: 87.5000 (85.9608)
Valid: 38 [ 250/390]  Loss: 1.749 (1.78)  Acc@1: 51.5625 (59.2256)  Acc@5: 89.0625 (86.0433)
Valid: 38 [ 300/390]  Loss: 1.877 (1.79)  Acc@1: 57.8125 (59.1362)  Acc@5: 82.8125 (85.9012)
Valid: 38 [ 350/390]  Loss: 1.391 (1.79)  Acc@1: 59.3750 (59.0946)  Acc@5: 85.9375 (85.9464)
Valid: 38 [ 390/390]  Loss: 1.903 (1.80)  Acc@1: 55.0000 (59.1560)  Acc@5: 87.5000 (85.8840)
valid_acc 59.156000
epoch = 38   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0564, 0.9436],
        [0.1180, 0.8820],
        [0.1955, 0.8045],
        [0.1488, 0.8512],
        [0.1254, 0.8746],
        [0.1102, 0.8898],
        [0.1336, 0.8664],
        [0.1401, 0.8599],
        [0.0497, 0.9503],
        [0.1816, 0.8184],
        [0.1379, 0.8621],
        [0.2493, 0.7507],
        [0.5070, 0.4930],
        [0.5223, 0.4777]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4697, 0.5303],
        [0.1866, 0.8134],
        [0.4139, 0.5861],
        [0.5058, 0.4942],
        [0.0667, 0.9333],
        [0.4422, 0.5578],
        [0.1277, 0.8723],
        [0.1201, 0.8799],
        [0.0712, 0.9288],
        [0.3650, 0.6350],
        [0.4792, 0.5208],
        [0.0623, 0.9377],
        [0.0606, 0.9394],
        [0.1728, 0.8272]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 39 [   0/390]  Loss: 0.3665 (0.366)  Acc@1: 89.0625 (89.0625)  Acc@5: 100.0000 (100.0000)LR: 3.754e-03
Train: 39 [  50/390]  Loss: 0.4136 (0.355)  Acc@1: 84.3750 (89.2157)  Acc@5: 98.4375 (99.2341)LR: 3.754e-03
Train: 39 [ 100/390]  Loss: 0.3655 (0.346)  Acc@1: 90.6250 (89.3410)  Acc@5: 98.4375 (99.2729)LR: 3.754e-03
Train: 39 [ 150/390]  Loss: 0.3060 (0.354)  Acc@1: 89.0625 (89.1246)  Acc@5: 100.0000 (99.2446)LR: 3.754e-03
Train: 39 [ 200/390]  Loss: 0.2675 (0.355)  Acc@1: 93.7500 (89.0780)  Acc@5: 100.0000 (99.2226)LR: 3.754e-03
Train: 39 [ 250/390]  Loss: 0.4045 (0.356)  Acc@1: 85.9375 (89.1061)  Acc@5: 98.4375 (99.2343)LR: 3.754e-03
Train: 39 [ 300/390]  Loss: 0.3307 (0.360)  Acc@1: 89.0625 (88.9483)  Acc@5: 100.0000 (99.2213)LR: 3.754e-03
Train: 39 [ 350/390]  Loss: 0.7081 (0.362)  Acc@1: 73.4375 (88.9334)  Acc@5: 98.4375 (99.2343)LR: 3.754e-03
Train: 39 [ 390/390]  Loss: 0.4633 (0.365)  Acc@1: 77.5000 (88.7440)  Acc@5: 100.0000 (99.2400)LR: 3.754e-03
train_acc 88.744000
Valid: 39 [   0/390]  Loss: 1.314 (1.31)  Acc@1: 68.7500 (68.7500)  Acc@5: 89.0625 (89.0625)
Valid: 39 [  50/390]  Loss: 1.602 (1.81)  Acc@1: 59.3750 (58.9154)  Acc@5: 85.9375 (84.8958)
Valid: 39 [ 100/390]  Loss: 1.449 (1.80)  Acc@1: 68.7500 (59.3595)  Acc@5: 89.0625 (85.4270)
Valid: 39 [ 150/390]  Loss: 1.948 (1.78)  Acc@1: 56.2500 (59.5923)  Acc@5: 84.3750 (85.6374)
Valid: 39 [ 200/390]  Loss: 1.785 (1.76)  Acc@1: 54.6875 (59.4916)  Acc@5: 87.5000 (85.9142)
Valid: 39 [ 250/390]  Loss: 1.968 (1.77)  Acc@1: 54.6875 (59.4871)  Acc@5: 82.8125 (85.7881)
Valid: 39 [ 300/390]  Loss: 2.453 (1.76)  Acc@1: 57.8125 (59.7124)  Acc@5: 82.8125 (85.9998)
Valid: 39 [ 350/390]  Loss: 1.563 (1.77)  Acc@1: 67.1875 (59.7979)  Acc@5: 89.0625 (86.0933)
Valid: 39 [ 390/390]  Loss: 1.849 (1.78)  Acc@1: 60.0000 (59.8160)  Acc@5: 85.0000 (86.0200)
valid_acc 59.816000
epoch = 39   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0534, 0.9466],
        [0.1157, 0.8843],
        [0.1941, 0.8059],
        [0.1490, 0.8510],
        [0.1228, 0.8772],
        [0.1038, 0.8962],
        [0.1326, 0.8674],
        [0.1401, 0.8599],
        [0.0482, 0.9518],
        [0.1789, 0.8211],
        [0.1299, 0.8701],
        [0.2516, 0.7484],
        [0.5068, 0.4932],
        [0.5225, 0.4775]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4653, 0.5347],
        [0.1819, 0.8181],
        [0.4106, 0.5894],
        [0.5026, 0.4974],
        [0.0643, 0.9357],
        [0.4356, 0.5644],
        [0.1223, 0.8777],
        [0.1147, 0.8853],
        [0.0680, 0.9320],
        [0.3593, 0.6407],
        [0.4753, 0.5247],
        [0.0596, 0.9404],
        [0.0569, 0.9431],
        [0.1702, 0.8298]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 40 [   0/390]  Loss: 0.2450 (0.245)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 3.292e-03
Train: 40 [  50/390]  Loss: 0.2066 (0.303)  Acc@1: 92.1875 (90.4105)  Acc@5: 100.0000 (99.6936)LR: 3.292e-03
Train: 40 [ 100/390]  Loss: 0.2630 (0.316)  Acc@1: 90.6250 (90.1764)  Acc@5: 100.0000 (99.6132)LR: 3.292e-03
Train: 40 [ 150/390]  Loss: 0.2104 (0.320)  Acc@1: 95.3125 (90.2939)  Acc@5: 100.0000 (99.5550)LR: 3.292e-03
Train: 40 [ 200/390]  Loss: 0.2229 (0.321)  Acc@1: 96.8750 (90.3529)  Acc@5: 100.0000 (99.5180)LR: 3.292e-03
Train: 40 [ 250/390]  Loss: 0.4200 (0.328)  Acc@1: 87.5000 (90.1270)  Acc@5: 98.4375 (99.4211)LR: 3.292e-03
Train: 40 [ 300/390]  Loss: 0.3562 (0.332)  Acc@1: 87.5000 (90.0021)  Acc@5: 100.0000 (99.3407)LR: 3.292e-03
Train: 40 [ 350/390]  Loss: 0.3924 (0.331)  Acc@1: 87.5000 (90.0240)  Acc@5: 98.4375 (99.3456)LR: 3.292e-03
Train: 40 [ 390/390]  Loss: 0.4796 (0.331)  Acc@1: 82.5000 (89.9800)  Acc@5: 95.0000 (99.3360)LR: 3.292e-03
train_acc 89.980000
Valid: 40 [   0/390]  Loss: 2.018 (2.02)  Acc@1: 56.2500 (56.2500)  Acc@5: 84.3750 (84.3750)
Valid: 40 [  50/390]  Loss: 1.797 (1.85)  Acc@1: 57.8125 (59.5282)  Acc@5: 82.8125 (85.3860)
Valid: 40 [ 100/390]  Loss: 2.174 (1.82)  Acc@1: 53.1250 (59.5297)  Acc@5: 85.9375 (85.9530)
Valid: 40 [ 150/390]  Loss: 1.401 (1.83)  Acc@1: 67.1875 (60.0062)  Acc@5: 89.0625 (85.9375)
Valid: 40 [ 200/390]  Loss: 1.796 (1.82)  Acc@1: 62.5000 (59.8181)  Acc@5: 84.3750 (86.1007)
Valid: 40 [ 250/390]  Loss: 2.141 (1.83)  Acc@1: 56.2500 (59.4746)  Acc@5: 79.6875 (85.9873)
Valid: 40 [ 300/390]  Loss: 1.819 (1.84)  Acc@1: 62.5000 (59.3646)  Acc@5: 87.5000 (85.9531)
Valid: 40 [ 350/390]  Loss: 1.616 (1.83)  Acc@1: 67.1875 (59.4507)  Acc@5: 81.2500 (85.9642)
Valid: 40 [ 390/390]  Loss: 1.827 (1.83)  Acc@1: 57.5000 (59.5880)  Acc@5: 87.5000 (86.0240)
valid_acc 59.588000
epoch = 40   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0509, 0.9491],
        [0.1138, 0.8862],
        [0.1898, 0.8102],
        [0.1467, 0.8533],
        [0.1187, 0.8813],
        [0.0998, 0.9002],
        [0.1319, 0.8681],
        [0.1393, 0.8607],
        [0.0467, 0.9533],
        [0.1766, 0.8234],
        [0.1219, 0.8781],
        [0.2535, 0.7465],
        [0.5082, 0.4918],
        [0.5248, 0.4752]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4654, 0.5346],
        [0.1785, 0.8215],
        [0.4101, 0.5899],
        [0.5025, 0.4975],
        [0.0629, 0.9371],
        [0.4328, 0.5672],
        [0.1190, 0.8810],
        [0.1109, 0.8891],
        [0.0664, 0.9336],
        [0.3545, 0.6455],
        [0.4711, 0.5289],
        [0.0573, 0.9427],
        [0.0550, 0.9450],
        [0.1662, 0.8338]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 41 [   0/390]  Loss: 0.3178 (0.318)  Acc@1: 87.5000 (87.5000)  Acc@5: 98.4375 (98.4375)LR: 2.868e-03
Train: 41 [  50/390]  Loss: 0.2504 (0.288)  Acc@1: 93.7500 (92.1569)  Acc@5: 98.4375 (99.4485)LR: 2.868e-03
Train: 41 [ 100/390]  Loss: 0.2702 (0.280)  Acc@1: 92.1875 (91.9090)  Acc@5: 100.0000 (99.4895)LR: 2.868e-03
Train: 41 [ 150/390]  Loss: 0.2463 (0.288)  Acc@1: 95.3125 (91.6598)  Acc@5: 100.0000 (99.4723)LR: 2.868e-03
Train: 41 [ 200/390]  Loss: 0.3174 (0.289)  Acc@1: 87.5000 (91.4723)  Acc@5: 100.0000 (99.5025)LR: 2.868e-03
Train: 41 [ 250/390]  Loss: 0.2795 (0.293)  Acc@1: 90.6250 (91.3347)  Acc@5: 100.0000 (99.4273)LR: 2.868e-03
Train: 41 [ 300/390]  Loss: 0.2975 (0.299)  Acc@1: 90.6250 (91.0974)  Acc@5: 100.0000 (99.4342)LR: 2.868e-03
Train: 41 [ 350/390]  Loss: 0.3116 (0.299)  Acc@1: 87.5000 (90.9678)  Acc@5: 100.0000 (99.4391)LR: 2.868e-03
Train: 41 [ 390/390]  Loss: 0.3482 (0.303)  Acc@1: 85.0000 (90.8640)  Acc@5: 97.5000 (99.4240)LR: 2.868e-03
train_acc 90.864000
Valid: 41 [   0/390]  Loss: 1.369 (1.37)  Acc@1: 67.1875 (67.1875)  Acc@5: 92.1875 (92.1875)
Valid: 41 [  50/390]  Loss: 1.575 (1.75)  Acc@1: 57.8125 (61.0294)  Acc@5: 84.3750 (86.4277)
Valid: 41 [ 100/390]  Loss: 1.771 (1.80)  Acc@1: 65.6250 (60.6436)  Acc@5: 82.8125 (86.0303)
Valid: 41 [ 150/390]  Loss: 1.858 (1.81)  Acc@1: 59.3750 (60.1200)  Acc@5: 85.9375 (86.1031)
Valid: 41 [ 200/390]  Loss: 1.581 (1.82)  Acc@1: 68.7500 (59.9502)  Acc@5: 87.5000 (86.0774)
Valid: 41 [ 250/390]  Loss: 1.314 (1.82)  Acc@1: 68.7500 (59.8232)  Acc@5: 92.1875 (85.9562)
Valid: 41 [ 300/390]  Loss: 2.332 (1.83)  Acc@1: 53.1250 (59.5775)  Acc@5: 82.8125 (85.8804)
Valid: 41 [ 350/390]  Loss: 1.885 (1.84)  Acc@1: 56.2500 (59.6644)  Acc@5: 81.2500 (85.8974)
Valid: 41 [ 390/390]  Loss: 1.989 (1.83)  Acc@1: 57.5000 (59.6360)  Acc@5: 85.0000 (85.9080)
valid_acc 59.636000
epoch = 41   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0492, 0.9508],
        [0.1111, 0.8889],
        [0.1853, 0.8147],
        [0.1479, 0.8521],
        [0.1150, 0.8850],
        [0.0946, 0.9054],
        [0.1317, 0.8683],
        [0.1381, 0.8619],
        [0.0457, 0.9543],
        [0.1723, 0.8277],
        [0.1141, 0.8859],
        [0.2551, 0.7449],
        [0.5124, 0.4876],
        [0.5264, 0.4736]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4645, 0.5355],
        [0.1754, 0.8246],
        [0.4098, 0.5902],
        [0.5063, 0.4937],
        [0.0607, 0.9393],
        [0.4318, 0.5682],
        [0.1165, 0.8835],
        [0.1088, 0.8912],
        [0.0649, 0.9351],
        [0.3510, 0.6490],
        [0.4690, 0.5310],
        [0.0560, 0.9440],
        [0.0533, 0.9467],
        [0.1645, 0.8355]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 42 [   0/390]  Loss: 0.1694 (0.169)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 2.484e-03
Train: 42 [  50/390]  Loss: 0.2164 (0.255)  Acc@1: 92.1875 (92.4326)  Acc@5: 100.0000 (99.6630)LR: 2.484e-03
Train: 42 [ 100/390]  Loss: 0.3395 (0.267)  Acc@1: 87.5000 (92.1875)  Acc@5: 100.0000 (99.5823)LR: 2.484e-03
Train: 42 [ 150/390]  Loss: 0.2023 (0.267)  Acc@1: 96.8750 (92.0944)  Acc@5: 100.0000 (99.5137)LR: 2.484e-03
Train: 42 [ 200/390]  Loss: 0.3791 (0.274)  Acc@1: 87.5000 (91.8144)  Acc@5: 100.0000 (99.5491)LR: 2.484e-03
Train: 42 [ 250/390]  Loss: 0.2914 (0.276)  Acc@1: 89.0625 (91.7766)  Acc@5: 100.0000 (99.5518)LR: 2.484e-03
Train: 42 [ 300/390]  Loss: 0.2418 (0.280)  Acc@1: 95.3125 (91.6788)  Acc@5: 100.0000 (99.5276)LR: 2.484e-03
Train: 42 [ 350/390]  Loss: 0.2752 (0.282)  Acc@1: 89.0625 (91.5910)  Acc@5: 100.0000 (99.5415)LR: 2.484e-03
Train: 42 [ 390/390]  Loss: 0.2658 (0.283)  Acc@1: 92.5000 (91.4960)  Acc@5: 97.5000 (99.5560)LR: 2.484e-03
train_acc 91.496000
Valid: 42 [   0/390]  Loss: 1.619 (1.62)  Acc@1: 68.7500 (68.7500)  Acc@5: 84.3750 (84.3750)
Valid: 42 [  50/390]  Loss: 1.816 (1.74)  Acc@1: 68.7500 (61.2439)  Acc@5: 87.5000 (86.0907)
Valid: 42 [ 100/390]  Loss: 1.875 (1.81)  Acc@1: 56.2500 (59.8700)  Acc@5: 84.3750 (85.5507)
Valid: 42 [ 150/390]  Loss: 1.786 (1.83)  Acc@1: 65.6250 (59.7475)  Acc@5: 89.0625 (85.3994)
Valid: 42 [ 200/390]  Loss: 2.057 (1.82)  Acc@1: 54.6875 (59.9036)  Acc@5: 82.8125 (85.5644)
Valid: 42 [ 250/390]  Loss: 1.501 (1.83)  Acc@1: 60.9375 (59.7796)  Acc@5: 89.0625 (85.7756)
Valid: 42 [ 300/390]  Loss: 1.785 (1.82)  Acc@1: 56.2500 (59.7539)  Acc@5: 85.9375 (85.8804)
Valid: 42 [ 350/390]  Loss: 2.152 (1.83)  Acc@1: 57.8125 (59.6777)  Acc@5: 82.8125 (85.8707)
Valid: 42 [ 390/390]  Loss: 1.302 (1.84)  Acc@1: 65.0000 (59.6320)  Acc@5: 90.0000 (85.8560)
valid_acc 59.632000
epoch = 42   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0474, 0.9526],
        [0.1093, 0.8907],
        [0.1828, 0.8172],
        [0.1471, 0.8529],
        [0.1126, 0.8874],
        [0.0903, 0.9097],
        [0.1322, 0.8678],
        [0.1352, 0.8648],
        [0.0446, 0.9554],
        [0.1703, 0.8297],
        [0.1066, 0.8934],
        [0.2546, 0.7454],
        [0.5169, 0.4831],
        [0.5323, 0.4677]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4607, 0.5393],
        [0.1725, 0.8275],
        [0.4070, 0.5930],
        [0.5048, 0.4952],
        [0.0594, 0.9406],
        [0.4302, 0.5698],
        [0.1135, 0.8865],
        [0.1047, 0.8953],
        [0.0620, 0.9380],
        [0.3480, 0.6520],
        [0.4648, 0.5352],
        [0.0544, 0.9456],
        [0.0514, 0.9486],
        [0.1635, 0.8365]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 43 [   0/390]  Loss: 0.3069 (0.307)  Acc@1: 90.6250 (90.6250)  Acc@5: 100.0000 (100.0000)LR: 2.142e-03
Train: 43 [  50/390]  Loss: 0.3717 (0.252)  Acc@1: 93.7500 (92.2181)  Acc@5: 96.8750 (99.7549)LR: 2.142e-03
Train: 43 [ 100/390]  Loss: 0.2809 (0.263)  Acc@1: 89.0625 (91.9709)  Acc@5: 100.0000 (99.6442)LR: 2.142e-03
Train: 43 [ 150/390]  Loss: 0.3798 (0.263)  Acc@1: 90.6250 (92.0426)  Acc@5: 98.4375 (99.6689)LR: 2.142e-03
Train: 43 [ 200/390]  Loss: 0.3441 (0.264)  Acc@1: 90.6250 (92.0631)  Acc@5: 98.4375 (99.6191)LR: 2.142e-03
Train: 43 [ 250/390]  Loss: 0.4204 (0.265)  Acc@1: 81.2500 (91.9011)  Acc@5: 100.0000 (99.6203)LR: 2.142e-03
Train: 43 [ 300/390]  Loss: 0.1838 (0.265)  Acc@1: 96.8750 (92.0214)  Acc@5: 100.0000 (99.6522)LR: 2.142e-03
Train: 43 [ 350/390]  Loss: 0.2555 (0.264)  Acc@1: 92.1875 (92.0851)  Acc@5: 98.4375 (99.6305)LR: 2.142e-03
Train: 43 [ 390/390]  Loss: 0.1327 (0.266)  Acc@1: 95.0000 (92.0480)  Acc@5: 100.0000 (99.6200)LR: 2.142e-03
train_acc 92.048000
Valid: 43 [   0/390]  Loss: 1.407 (1.41)  Acc@1: 67.1875 (67.1875)  Acc@5: 90.6250 (90.6250)
Valid: 43 [  50/390]  Loss: 1.541 (1.89)  Acc@1: 71.8750 (59.6201)  Acc@5: 90.6250 (85.5699)
Valid: 43 [ 100/390]  Loss: 2.070 (1.83)  Acc@1: 59.3750 (60.3187)  Acc@5: 84.3750 (85.9375)
Valid: 43 [ 150/390]  Loss: 1.720 (1.82)  Acc@1: 54.6875 (60.0579)  Acc@5: 87.5000 (86.0617)
Valid: 43 [ 200/390]  Loss: 1.744 (1.81)  Acc@1: 59.3750 (60.1990)  Acc@5: 92.1875 (86.2484)
Valid: 43 [ 250/390]  Loss: 1.865 (1.82)  Acc@1: 53.1250 (59.9228)  Acc@5: 90.6250 (86.1554)
Valid: 43 [ 300/390]  Loss: 2.242 (1.84)  Acc@1: 54.6875 (59.9356)  Acc@5: 79.6875 (86.0154)
Valid: 43 [ 350/390]  Loss: 1.975 (1.84)  Acc@1: 57.8125 (59.9448)  Acc@5: 89.0625 (85.9909)
Valid: 43 [ 390/390]  Loss: 1.619 (1.84)  Acc@1: 60.0000 (59.7320)  Acc@5: 87.5000 (85.9240)
valid_acc 59.732000
epoch = 43   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0454, 0.9546],
        [0.1083, 0.8917],
        [0.1799, 0.8201],
        [0.1471, 0.8529],
        [0.1109, 0.8891],
        [0.0864, 0.9136],
        [0.1333, 0.8667],
        [0.1333, 0.8667],
        [0.0437, 0.9563],
        [0.1666, 0.8334],
        [0.1011, 0.8989],
        [0.2534, 0.7466],
        [0.5226, 0.4774],
        [0.5380, 0.4620]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4556, 0.5444],
        [0.1686, 0.8314],
        [0.4034, 0.5966],
        [0.5041, 0.4959],
        [0.0578, 0.9422],
        [0.4238, 0.5762],
        [0.1101, 0.8899],
        [0.1020, 0.8980],
        [0.0606, 0.9394],
        [0.3412, 0.6588],
        [0.4631, 0.5369],
        [0.0533, 0.9467],
        [0.0497, 0.9503],
        [0.1589, 0.8411]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 44 [   0/390]  Loss: 0.3688 (0.369)  Acc@1: 85.9375 (85.9375)  Acc@5: 100.0000 (100.0000)LR: 1.843e-03
Train: 44 [  50/390]  Loss: 0.2884 (0.231)  Acc@1: 92.1875 (93.5355)  Acc@5: 100.0000 (99.7549)LR: 1.843e-03
Train: 44 [ 100/390]  Loss: 0.3323 (0.239)  Acc@1: 92.1875 (93.2395)  Acc@5: 98.4375 (99.6442)LR: 1.843e-03
Train: 44 [ 150/390]  Loss: 0.2761 (0.242)  Acc@1: 92.1875 (93.0981)  Acc@5: 98.4375 (99.6482)LR: 1.843e-03
Train: 44 [ 200/390]  Loss: 0.1965 (0.239)  Acc@1: 96.8750 (93.1048)  Acc@5: 100.0000 (99.6735)LR: 1.843e-03
Train: 44 [ 250/390]  Loss: 0.1601 (0.240)  Acc@1: 96.8750 (92.9843)  Acc@5: 100.0000 (99.7136)LR: 1.843e-03
Train: 44 [ 300/390]  Loss: 0.2584 (0.244)  Acc@1: 93.7500 (92.8312)  Acc@5: 98.4375 (99.6937)LR: 1.843e-03
Train: 44 [ 350/390]  Loss: 0.2866 (0.246)  Acc@1: 93.7500 (92.7528)  Acc@5: 98.4375 (99.6884)LR: 1.843e-03
Train: 44 [ 390/390]  Loss: 0.4314 (0.247)  Acc@1: 80.0000 (92.7160)  Acc@5: 97.5000 (99.6760)LR: 1.843e-03
train_acc 92.716000
Valid: 44 [   0/390]  Loss: 1.800 (1.80)  Acc@1: 60.9375 (60.9375)  Acc@5: 89.0625 (89.0625)
Valid: 44 [  50/390]  Loss: 1.828 (1.81)  Acc@1: 53.1250 (59.9877)  Acc@5: 89.0625 (86.7953)
Valid: 44 [ 100/390]  Loss: 1.547 (1.87)  Acc@1: 62.5000 (59.9319)  Acc@5: 90.6250 (86.2933)
Valid: 44 [ 150/390]  Loss: 1.747 (1.88)  Acc@1: 65.6250 (59.8820)  Acc@5: 85.9375 (86.1134)
Valid: 44 [ 200/390]  Loss: 1.669 (1.85)  Acc@1: 60.9375 (60.3933)  Acc@5: 89.0625 (86.2718)
Valid: 44 [ 250/390]  Loss: 1.354 (1.85)  Acc@1: 64.0625 (60.4644)  Acc@5: 92.1875 (86.4417)
Valid: 44 [ 300/390]  Loss: 1.820 (1.86)  Acc@1: 48.4375 (60.2886)  Acc@5: 90.6250 (86.3632)
Valid: 44 [ 350/390]  Loss: 2.051 (1.86)  Acc@1: 62.5000 (60.2074)  Acc@5: 85.9375 (86.3203)
Valid: 44 [ 390/390]  Loss: 2.066 (1.87)  Acc@1: 50.0000 (60.0000)  Acc@5: 82.5000 (86.2960)
valid_acc 60.000000
epoch = 44   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0438, 0.9562],
        [0.1057, 0.8943],
        [0.1781, 0.8219],
        [0.1451, 0.8549],
        [0.1088, 0.8912],
        [0.0826, 0.9174],
        [0.1318, 0.8682],
        [0.1324, 0.8676],
        [0.0429, 0.9571],
        [0.1630, 0.8370],
        [0.0956, 0.9044],
        [0.2538, 0.7462],
        [0.5255, 0.4745],
        [0.5456, 0.4544]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4541, 0.5459],
        [0.1634, 0.8366],
        [0.4018, 0.5982],
        [0.4996, 0.5004],
        [0.0564, 0.9436],
        [0.4221, 0.5779],
        [0.1068, 0.8932],
        [0.0995, 0.9005],
        [0.0594, 0.9406],
        [0.3382, 0.6618],
        [0.4603, 0.5397],
        [0.0514, 0.9486],
        [0.0479, 0.9521],
        [0.1557, 0.8443]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 45 [   0/390]  Loss: 0.2219 (0.222)  Acc@1: 93.7500 (93.7500)  Acc@5: 100.0000 (100.0000)LR: 1.587e-03
Train: 45 [  50/390]  Loss: 0.1911 (0.220)  Acc@1: 96.8750 (93.8113)  Acc@5: 100.0000 (99.6936)LR: 1.587e-03
Train: 45 [ 100/390]  Loss: 0.2845 (0.229)  Acc@1: 89.0625 (93.4406)  Acc@5: 100.0000 (99.6442)LR: 1.587e-03
Train: 45 [ 150/390]  Loss: 0.3584 (0.230)  Acc@1: 90.6250 (93.4189)  Acc@5: 96.8750 (99.6378)LR: 1.587e-03
Train: 45 [ 200/390]  Loss: 0.2551 (0.231)  Acc@1: 93.7500 (93.4546)  Acc@5: 100.0000 (99.6424)LR: 1.587e-03
Train: 45 [ 250/390]  Loss: 0.1833 (0.233)  Acc@1: 93.7500 (93.3329)  Acc@5: 100.0000 (99.6763)LR: 1.587e-03
Train: 45 [ 300/390]  Loss: 0.3037 (0.233)  Acc@1: 92.1875 (93.3243)  Acc@5: 98.4375 (99.6833)LR: 1.587e-03
Train: 45 [ 350/390]  Loss: 0.2946 (0.233)  Acc@1: 92.1875 (93.3360)  Acc@5: 98.4375 (99.6572)LR: 1.587e-03
Train: 45 [ 390/390]  Loss: 0.3346 (0.233)  Acc@1: 90.0000 (93.2400)  Acc@5: 100.0000 (99.6600)LR: 1.587e-03
train_acc 93.240000
Valid: 45 [   0/390]  Loss: 2.035 (2.04)  Acc@1: 62.5000 (62.5000)  Acc@5: 90.6250 (90.6250)
Valid: 45 [  50/390]  Loss: 2.424 (1.83)  Acc@1: 56.2500 (60.4473)  Acc@5: 78.1250 (86.2745)
Valid: 45 [ 100/390]  Loss: 1.744 (1.83)  Acc@1: 54.6875 (60.7673)  Acc@5: 87.5000 (86.6955)
Valid: 45 [ 150/390]  Loss: 1.711 (1.84)  Acc@1: 59.3750 (60.6478)  Acc@5: 87.5000 (86.4031)
Valid: 45 [ 200/390]  Loss: 2.738 (1.87)  Acc@1: 53.1250 (60.3933)  Acc@5: 81.2500 (86.2562)
Valid: 45 [ 250/390]  Loss: 1.896 (1.86)  Acc@1: 64.0625 (60.4582)  Acc@5: 85.9375 (86.2612)
Valid: 45 [ 300/390]  Loss: 2.033 (1.85)  Acc@1: 57.8125 (60.3873)  Acc@5: 82.8125 (86.2022)
Valid: 45 [ 350/390]  Loss: 1.428 (1.86)  Acc@1: 65.6250 (60.2431)  Acc@5: 90.6250 (86.2491)
Valid: 45 [ 390/390]  Loss: 1.739 (1.86)  Acc@1: 60.0000 (60.2040)  Acc@5: 92.5000 (86.2360)
valid_acc 60.204000
epoch = 45   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0428, 0.9572],
        [0.1046, 0.8954],
        [0.1774, 0.8226],
        [0.1448, 0.8552],
        [0.1063, 0.8937],
        [0.0790, 0.9210],
        [0.1302, 0.8698],
        [0.1305, 0.8695],
        [0.0422, 0.9578],
        [0.1597, 0.8403],
        [0.0902, 0.9098],
        [0.2522, 0.7478],
        [0.5316, 0.4684],
        [0.5477, 0.4523]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4545, 0.5455],
        [0.1598, 0.8402],
        [0.4025, 0.5975],
        [0.5010, 0.4990],
        [0.0546, 0.9454],
        [0.4258, 0.5742],
        [0.1035, 0.8965],
        [0.0973, 0.9027],
        [0.0586, 0.9414],
        [0.3399, 0.6601],
        [0.4635, 0.5365],
        [0.0502, 0.9498],
        [0.0471, 0.9529],
        [0.1532, 0.8468]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 46 [   0/390]  Loss: 0.2127 (0.213)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 1.377e-03
Train: 46 [  50/390]  Loss: 0.1702 (0.211)  Acc@1: 96.8750 (94.3934)  Acc@5: 100.0000 (99.7855)LR: 1.377e-03
Train: 46 [ 100/390]  Loss: 0.2256 (0.214)  Acc@1: 93.7500 (94.1058)  Acc@5: 100.0000 (99.7679)LR: 1.377e-03
Train: 46 [ 150/390]  Loss: 0.2025 (0.212)  Acc@1: 96.8750 (94.0708)  Acc@5: 100.0000 (99.7827)LR: 1.377e-03
Train: 46 [ 200/390]  Loss: 0.1619 (0.212)  Acc@1: 96.8750 (94.0376)  Acc@5: 100.0000 (99.7823)LR: 1.377e-03
Train: 46 [ 250/390]  Loss: 0.09131 (0.210)  Acc@1: 100.0000 (94.0426)  Acc@5: 100.0000 (99.7946)LR: 1.377e-03
Train: 46 [ 300/390]  Loss: 0.1484 (0.212)  Acc@1: 96.8750 (93.9784)  Acc@5: 100.0000 (99.7872)LR: 1.377e-03
Train: 46 [ 350/390]  Loss: 0.1608 (0.214)  Acc@1: 96.8750 (93.9726)  Acc@5: 100.0000 (99.7774)LR: 1.377e-03
Train: 46 [ 390/390]  Loss: 0.2010 (0.215)  Acc@1: 92.5000 (93.8760)  Acc@5: 100.0000 (99.7760)LR: 1.377e-03
train_acc 93.876000
Valid: 46 [   0/390]  Loss: 2.139 (2.14)  Acc@1: 60.9375 (60.9375)  Acc@5: 85.9375 (85.9375)
Valid: 46 [  50/390]  Loss: 2.658 (1.87)  Acc@1: 54.6875 (60.9069)  Acc@5: 79.6875 (86.0907)
Valid: 46 [ 100/390]  Loss: 2.205 (1.89)  Acc@1: 51.5625 (59.8855)  Acc@5: 84.3750 (85.7828)
Valid: 46 [ 150/390]  Loss: 2.469 (1.92)  Acc@1: 54.6875 (59.5509)  Acc@5: 84.3750 (85.3166)
Valid: 46 [ 200/390]  Loss: 1.598 (1.89)  Acc@1: 59.3750 (59.5538)  Acc@5: 92.1875 (85.7976)
Valid: 46 [ 250/390]  Loss: 1.521 (1.89)  Acc@1: 62.5000 (59.7610)  Acc@5: 90.6250 (85.7819)
Valid: 46 [ 300/390]  Loss: 1.412 (1.89)  Acc@1: 60.9375 (59.6605)  Acc@5: 90.6250 (85.8544)
Valid: 46 [ 350/390]  Loss: 2.139 (1.89)  Acc@1: 59.3750 (59.6421)  Acc@5: 84.3750 (85.9019)
Valid: 46 [ 390/390]  Loss: 1.661 (1.89)  Acc@1: 55.0000 (59.5200)  Acc@5: 90.0000 (86.0000)
valid_acc 59.520000
epoch = 46   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0414, 0.9586],
        [0.1028, 0.8972],
        [0.1761, 0.8239],
        [0.1435, 0.8566],
        [0.1055, 0.8945],
        [0.0761, 0.9239],
        [0.1276, 0.8724],
        [0.1291, 0.8709],
        [0.0414, 0.9586],
        [0.1575, 0.8425],
        [0.0866, 0.9134],
        [0.2517, 0.7483],
        [0.5302, 0.4698],
        [0.5574, 0.4426]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4509, 0.5491],
        [0.1566, 0.8434],
        [0.3994, 0.6006],
        [0.4939, 0.5061],
        [0.0535, 0.9465],
        [0.4245, 0.5755],
        [0.1007, 0.8993],
        [0.0952, 0.9048],
        [0.0578, 0.9422],
        [0.3352, 0.6648],
        [0.4588, 0.5412],
        [0.0484, 0.9516],
        [0.0450, 0.9550],
        [0.1498, 0.8502]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 47 [   0/390]  Loss: 0.2662 (0.266)  Acc@1: 96.8750 (96.8750)  Acc@5: 100.0000 (100.0000)LR: 1.213e-03
Train: 47 [  50/390]  Loss: 0.3198 (0.211)  Acc@1: 95.3125 (94.6078)  Acc@5: 98.4375 (99.7243)LR: 1.213e-03
Train: 47 [ 100/390]  Loss: 0.2506 (0.205)  Acc@1: 90.6250 (94.5235)  Acc@5: 100.0000 (99.8453)LR: 1.213e-03
Train: 47 [ 150/390]  Loss: 0.1646 (0.202)  Acc@1: 95.3125 (94.6089)  Acc@5: 100.0000 (99.8241)LR: 1.213e-03
Train: 47 [ 200/390]  Loss: 0.1289 (0.204)  Acc@1: 96.8750 (94.5507)  Acc@5: 100.0000 (99.8212)LR: 1.213e-03
Train: 47 [ 250/390]  Loss: 0.1786 (0.206)  Acc@1: 95.3125 (94.4659)  Acc@5: 100.0000 (99.8195)LR: 1.213e-03
Train: 47 [ 300/390]  Loss: 0.2220 (0.204)  Acc@1: 92.1875 (94.4508)  Acc@5: 100.0000 (99.8339)LR: 1.213e-03
Train: 47 [ 350/390]  Loss: 0.2892 (0.207)  Acc@1: 92.1875 (94.3287)  Acc@5: 98.4375 (99.8264)LR: 1.213e-03
Train: 47 [ 390/390]  Loss: 0.3800 (0.207)  Acc@1: 90.0000 (94.2840)  Acc@5: 97.5000 (99.8200)LR: 1.213e-03
train_acc 94.284000
Valid: 47 [   0/390]  Loss: 1.731 (1.73)  Acc@1: 64.0625 (64.0625)  Acc@5: 90.6250 (90.6250)
Valid: 47 [  50/390]  Loss: 1.646 (1.79)  Acc@1: 62.5000 (61.5196)  Acc@5: 84.3750 (85.7230)
Valid: 47 [ 100/390]  Loss: 1.995 (1.83)  Acc@1: 67.1875 (60.3960)  Acc@5: 84.3750 (85.5507)
Valid: 47 [ 150/390]  Loss: 1.370 (1.84)  Acc@1: 70.3125 (60.3063)  Acc@5: 89.0625 (85.8547)
Valid: 47 [ 200/390]  Loss: 1.548 (1.84)  Acc@1: 64.0625 (60.4322)  Acc@5: 92.1875 (85.9686)
Valid: 47 [ 250/390]  Loss: 1.491 (1.85)  Acc@1: 59.3750 (60.2403)  Acc@5: 90.6250 (85.9250)
Valid: 47 [ 300/390]  Loss: 1.811 (1.87)  Acc@1: 68.7500 (60.1692)  Acc@5: 89.0625 (85.8077)
Valid: 47 [ 350/390]  Loss: 1.564 (1.88)  Acc@1: 54.6875 (60.2564)  Acc@5: 93.7500 (85.8173)
Valid: 47 [ 390/390]  Loss: 1.194 (1.88)  Acc@1: 62.5000 (60.1440)  Acc@5: 92.5000 (85.7520)
valid_acc 60.144000
epoch = 47   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0399, 0.9601],
        [0.1023, 0.8977],
        [0.1726, 0.8274],
        [0.1430, 0.8570],
        [0.1044, 0.8956],
        [0.0728, 0.9272],
        [0.1267, 0.8733],
        [0.1297, 0.8703],
        [0.0408, 0.9592],
        [0.1527, 0.8473],
        [0.0818, 0.9182],
        [0.2516, 0.7484],
        [0.5313, 0.4687],
        [0.5640, 0.4360]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4524, 0.5476],
        [0.1523, 0.8477],
        [0.3979, 0.6021],
        [0.4886, 0.5114],
        [0.0525, 0.9475],
        [0.4236, 0.5764],
        [0.0985, 0.9015],
        [0.0929, 0.9071],
        [0.0566, 0.9434],
        [0.3326, 0.6674],
        [0.4563, 0.5437],
        [0.0469, 0.9531],
        [0.0441, 0.9559],
        [0.1458, 0.8542]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 48 [   0/390]  Loss: 0.2345 (0.234)  Acc@1: 90.6250 (90.6250)  Acc@5: 98.4375 (98.4375)LR: 1.095e-03
Train: 48 [  50/390]  Loss: 0.1734 (0.190)  Acc@1: 93.7500 (95.1287)  Acc@5: 100.0000 (99.8468)LR: 1.095e-03
Train: 48 [ 100/390]  Loss: 0.1185 (0.193)  Acc@1: 98.4375 (94.8639)  Acc@5: 100.0000 (99.7989)LR: 1.095e-03
Train: 48 [ 150/390]  Loss: 0.3226 (0.191)  Acc@1: 87.5000 (94.9400)  Acc@5: 100.0000 (99.8551)LR: 1.095e-03
Train: 48 [ 200/390]  Loss: 0.1997 (0.195)  Acc@1: 93.7500 (94.6906)  Acc@5: 100.0000 (99.8368)LR: 1.095e-03
Train: 48 [ 250/390]  Loss: 0.2349 (0.196)  Acc@1: 93.7500 (94.6962)  Acc@5: 96.8750 (99.8070)LR: 1.095e-03
Train: 48 [ 300/390]  Loss: 0.1660 (0.199)  Acc@1: 96.8750 (94.5390)  Acc@5: 100.0000 (99.8079)LR: 1.095e-03
Train: 48 [ 350/390]  Loss: 0.2658 (0.200)  Acc@1: 90.6250 (94.4934)  Acc@5: 100.0000 (99.8219)LR: 1.095e-03
Train: 48 [ 390/390]  Loss: 0.1780 (0.199)  Acc@1: 100.0000 (94.5400)  Acc@5: 100.0000 (99.8320)LR: 1.095e-03
train_acc 94.540000
Valid: 48 [   0/390]  Loss: 1.716 (1.72)  Acc@1: 59.3750 (59.3750)  Acc@5: 85.9375 (85.9375)
Valid: 48 [  50/390]  Loss: 1.785 (1.83)  Acc@1: 68.7500 (60.8150)  Acc@5: 84.3750 (86.6728)
Valid: 48 [ 100/390]  Loss: 1.429 (1.85)  Acc@1: 64.0625 (60.5043)  Acc@5: 87.5000 (86.3861)
Valid: 48 [ 150/390]  Loss: 2.309 (1.87)  Acc@1: 45.3125 (60.2546)  Acc@5: 84.3750 (86.3100)
Valid: 48 [ 200/390]  Loss: 1.545 (1.86)  Acc@1: 60.9375 (60.3856)  Acc@5: 92.1875 (86.3106)
Valid: 48 [ 250/390]  Loss: 2.084 (1.87)  Acc@1: 54.6875 (60.1780)  Acc@5: 78.1250 (86.3484)
Valid: 48 [ 300/390]  Loss: 1.847 (1.89)  Acc@1: 67.1875 (59.8578)  Acc@5: 79.6875 (85.9635)
Valid: 48 [ 350/390]  Loss: 1.804 (1.89)  Acc@1: 56.2500 (59.9715)  Acc@5: 85.9375 (85.9108)
Valid: 48 [ 390/390]  Loss: 1.773 (1.90)  Acc@1: 57.5000 (59.9280)  Acc@5: 95.0000 (85.8200)
valid_acc 59.928000
epoch = 48   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0389, 0.9611],
        [0.1007, 0.8993],
        [0.1713, 0.8287],
        [0.1423, 0.8577],
        [0.1043, 0.8957],
        [0.0709, 0.9291],
        [0.1265, 0.8735],
        [0.1295, 0.8705],
        [0.0403, 0.9597],
        [0.1487, 0.8513],
        [0.0790, 0.9210],
        [0.2492, 0.7508],
        [0.5316, 0.4684],
        [0.5689, 0.4311]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4469, 0.5531],
        [0.1497, 0.8503],
        [0.3965, 0.6035],
        [0.4895, 0.5105],
        [0.0519, 0.9481],
        [0.4202, 0.5798],
        [0.0966, 0.9034],
        [0.0913, 0.9087],
        [0.0554, 0.9446],
        [0.3293, 0.6707],
        [0.4565, 0.5435],
        [0.0463, 0.9537],
        [0.0431, 0.9569],
        [0.1433, 0.8567]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
Train: 49 [   0/390]  Loss: 0.1757 (0.176)  Acc@1: 95.3125 (95.3125)  Acc@5: 100.0000 (100.0000)LR: 1.024e-03
Train: 49 [  50/390]  Loss: 0.2400 (0.182)  Acc@1: 92.1875 (94.9142)  Acc@5: 100.0000 (99.8775)LR: 1.024e-03
Train: 49 [ 100/390]  Loss: 0.2452 (0.190)  Acc@1: 93.7500 (94.6163)  Acc@5: 100.0000 (99.8298)LR: 1.024e-03
Train: 49 [ 150/390]  Loss: 0.1942 (0.193)  Acc@1: 93.7500 (94.6813)  Acc@5: 100.0000 (99.8137)LR: 1.024e-03
Train: 49 [ 200/390]  Loss: 0.1168 (0.192)  Acc@1: 98.4375 (94.6673)  Acc@5: 100.0000 (99.8134)LR: 1.024e-03
Train: 49 [ 250/390]  Loss: 0.2625 (0.193)  Acc@1: 92.1875 (94.6153)  Acc@5: 98.4375 (99.8195)LR: 1.024e-03
Train: 49 [ 300/390]  Loss: 0.1636 (0.195)  Acc@1: 95.3125 (94.6688)  Acc@5: 100.0000 (99.8079)LR: 1.024e-03
Train: 49 [ 350/390]  Loss: 0.2052 (0.193)  Acc@1: 98.4375 (94.7828)  Acc@5: 100.0000 (99.8219)LR: 1.024e-03
Train: 49 [ 390/390]  Loss: 0.2790 (0.194)  Acc@1: 90.0000 (94.6680)  Acc@5: 100.0000 (99.8320)LR: 1.024e-03
train_acc 94.668000
Valid: 49 [   0/390]  Loss: 1.701 (1.70)  Acc@1: 67.1875 (67.1875)  Acc@5: 85.9375 (85.9375)
Valid: 49 [  50/390]  Loss: 1.811 (1.96)  Acc@1: 59.3750 (58.4865)  Acc@5: 81.2500 (85.5699)
Valid: 49 [ 100/390]  Loss: 2.117 (1.90)  Acc@1: 73.4375 (60.0402)  Acc@5: 84.3750 (85.9839)
Valid: 49 [ 150/390]  Loss: 1.881 (1.88)  Acc@1: 57.8125 (59.9752)  Acc@5: 87.5000 (86.0720)
Valid: 49 [ 200/390]  Loss: 2.377 (1.89)  Acc@1: 59.3750 (60.1679)  Acc@5: 78.1250 (85.9297)
Valid: 49 [ 250/390]  Loss: 2.680 (1.89)  Acc@1: 51.5625 (60.1594)  Acc@5: 78.1250 (85.7943)
Valid: 49 [ 300/390]  Loss: 1.724 (1.88)  Acc@1: 60.9375 (60.0031)  Acc@5: 87.5000 (85.9738)
Valid: 49 [ 350/390]  Loss: 1.923 (1.89)  Acc@1: 57.8125 (59.9982)  Acc@5: 82.8125 (85.8307)
Valid: 49 [ 390/390]  Loss: 1.343 (1.89)  Acc@1: 67.5000 (60.0280)  Acc@5: 92.5000 (85.9240)
valid_acc 60.028000
epoch = 49   
 genotype = Genotype(normal=[('dil_conv_3x3', 0), ('dil_conv_5x5', 1), ('dil_conv_3x3', 2), ('sep_conv_3x3', 1), ('dil_conv_3x3', 3), ('skip_connect', 0), ('skip_connect', 1), ('sep_conv_3x3', 0)], normal_concat=range(2, 6), reduce=[('dil_conv_3x3', 1), ('avg_pool_3x3', 0), ('dil_conv_5x5', 2), ('avg_pool_3x3', 0), ('dil_conv_5x5', 3), ('dil_conv_3x3', 2), ('dil_conv_5x5', 3), ('dil_conv_5x5', 2)], reduce_concat=range(2, 6))
alphas_normal = 
 tensor([[0.0377, 0.9623],
        [0.0979, 0.9021],
        [0.1691, 0.8309],
        [0.1428, 0.8572],
        [0.1026, 0.8974],
        [0.0686, 0.9314],
        [0.1238, 0.8762],
        [0.1298, 0.8702],
        [0.0397, 0.9603],
        [0.1462, 0.8538],
        [0.0764, 0.9236],
        [0.2499, 0.7501],
        [0.5293, 0.4707],
        [0.5740, 0.4260]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
 alphas_reduct = 
 tensor([[0.4435, 0.5565],
        [0.1469, 0.8531],
        [0.3973, 0.6027],
        [0.4848, 0.5152],
        [0.0510, 0.9490],
        [0.4183, 0.5817],
        [0.0942, 0.9058],
        [0.0885, 0.9115],
        [0.0541, 0.9459],
        [0.3259, 0.6741],
        [0.4529, 0.5471],
        [0.0454, 0.9546],
        [0.0420, 0.9580],
        [0.1388, 0.8612]], device='cuda:0', grad_fn=<SoftmaxBackward0>)
